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Executive Summary 


Effective international S&E partnerships advance the S&E enterprise and energize U.S. 
innovation and economic competitiveness, but they also have great potential to improve 
relations among countries and regions and to build greater S&E capacity around the world . 1 

Over a decade ago, the National Science Board (NSB) highlighted the importance of international 
collaboration in its call for increased government commitment to promoting international science 
and engineering (S&E) research and education. 2 The NSB also identified the National Science 
Foundation (NSF) as having an important leadership role in international S&E research and 
education activities, specifically in promoting international S&E among early career scientists and 
engineers. 3 

Among NSF's postdoctoral programs, the International Research Fellowship Program (IRFP) is unique 
in its emphasis on providing postdoctoral fellows with international research experiences. 4 
Established in 1992, IRFP provides financial support to postdoctoral scientists for a research 
experience abroad lasting from 9 to 24 months; 5 there is no restriction on the geographical location 
of the hosting institution. IRFP aligns closely with the NSB's call for NSF to support the international 
science engagement of scientists and engineers. 

NSF contracted with Abt Associates to conduct an evaluation of IRFP, to gather evidence about 
whether the IRFP program has achieved its goal of furthering the collaborative activities and 
international partnerships of early career STEM researchers. The evaluation found evidence that the 
IRFP program is meeting its goals, which are to: 

• Introduce early career scientists and engineers to opportunities for international research 
collaboration; 

• Build research capacity and global perspective of participants; and 

• Forge long-term relationships between U.S. and foreign S&E researchers. 

This report presents the findings from the evaluation of IRFP, which was designed to investigate the 
characteristics of IRFP applicants and their motivations for participation, the host researchers' 


1 National Science Board. 2008. International Science and Engineering Partnerships: A Priority for U.S. 
Foreign Policy and Our Nation's Innovation Enterprise. NSB-08-4. Arlington, VA: National Science 
Foundation, p. 1 

2 NSB. 2001. Toward a More Effective Role for the U.S. Government in International Science and Engineering 
NSB-01-187. Arlington, VA: National Science Foundation. 

3 NSB. 2000. Toward a More Effective NSF Role in International Science and Engineering. NSB-00-217. 
Arlington, VA: National Science Foundation. 

4 A list and descriptions of NSF postdoctoral opportunities can be found at 
http://www.nsf. gov/funding/education. jsp?fund_type=3 

5 National Science Foundation. 2006a. IRFP Program Solicitation. Retrieved from 
http://www.nsf.gOv/pubs/2006/nsf06582/nsf06582.html#pgm_desc_txt 
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motivations for participating in the program, participants' experiences in and perceptions of the 
program, and outcomes of the program. 

The importance of international collaboration cannot be overstated. Researchers and policymakers 
agree that while the United States remains a leader in S&E research and development, the nation 
cannot maintain this position without engaging in international collaborations. Promoting 
international engagement at all levels is crucial to fostering successful research partnerships and 
developing the next generation of S&E researchers. Providing early career researchers with an 
opportunity to engage in an international research experience may help them improve their own 
research capabilities and pursue future collaborations with international colleagues. This, in turn, 
could lead U.S. researchers to reap benefits such as increased visibility in the research community, 
access to more substantial funding and resources, and the opportunity to benefit from the expertise 
of international peers. 

Findings from this evaluation demonstrate that individuals derive important benefits from the 
program, both on an individual and a collective basis. Former fellows and hosts strongly endorsed 
the IRFP program, and would recommend IRFP to their students and colleagues. IRFP offers an 
opportunity for interested researchers to develop their international research collaborations early in 
their careers. The experiences during IRFP seed relationships that often are sustained and that 
generate international research collaborations across geopolitical boundaries. As such, IRFP has a 
central role in NSF's efforts to respond the NSB's charge that NSF play a leadership role in 
international S&E research and education activities, and specifically in promoting international S&E 
among early career scientists and engineers. 


Key Findings 


To align with the program goals, the evaluation paid particular attention to the opportunities for 
international research collaborations, the experiences to develop research capacity and global 
perspectives, and the relationships between U.S .and foreign researchers. Specifically, the 
evaluation examined the following questions: 

1. Does the extent to which former fellows engage in international collaborations differ from 
those of unfunded applicants? 

2. Do fellows' post-award career activities and job characteristics differ from unfunded 
applicants? 

3. What are the perceived outcomes of program participation? 

4. Do the outcomes of program participation extend beyond the direct participants? 

The evaluation provides evidence that the IRFP experience leads to greater levels of international 
research engagement among fellows. 

• There are statistically significant and positive differences between fellows and unfunded 
applicants on the number of international postdoctoral fellowships held. 
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• IRFP fellows had a larger number of publications with a foreign co-author compared to non- 
funded applicants, and the percentage of publications with a foreign co-author was also 
greater for fellows. These differences were statistically significant. 

Importantly, this international focus does not come at the expense of research productivity or 
professional advancement. 

• Fellows and their peers were equally likely to hold multiple postdoctoral appointments, and 
were equally productive researchers, equally likely to hold a faculty rank of assistant, 
associate, or full professor, and equally likely to be tenured. 

• Career outcomes of IRFP fellows, and applicants overall, compare well against national 
STEM PhD holders on employment, publications, and international collaborations, 
suggesting that IRFP attracts a talented pool of applicants. 

Former IRFP fellows also reported that they reaped positive career outcomes from their 
participation in IRFP. 

• About four-fifths of fellows (79 percent) reported that participating in IRFP had qualified 
them for a broader range of career options, and 68 percent said that IRFP had made them 
more competitive for jobs. 

• Fellows reported that IRFP had opened up new areas for investigation (71 percent). 

The relationships developed during IRFP seed subsequent professional collaborations and activities 
that endure beyond the fellowship period. 

• Nearly three-quarters (71 percent) of IRFP fellows reported that they had made important 
connections to researchers in their host country. 

• Most fellows reported that the fellowships offer opportunities for professional relationships 
that endure beyond the fellowship period, either through subsequent collaborations with 
their hosts, and/or additional communications (46 percent each). Half of all former fellows 
said that participation in IRFP made them more committed to international research 
collaboration. 

Further, the program demonstrates potential to reach beyond the immediate participants. 

• More than three-quarters of former IRFP fellows reported that post-fellowship, they taught 
colleagues, students or peers methods learned during their fellowship (78 percent), and 
shared resources or tools acquired during this time (75 percent). 

• Flosts also served as a mechanism for extending the effects of the program to other 
scientists. Twenty-five percent of former IRFP hosts reported that a collaboration with a U.S. 
researcher resulted from their participation in the IRFP program. 
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Additional Findings 


The evaluation was also designed to address the following questions: 

5. What are the characteristics of people who apply for and participate in the IRFP program? 

6. What motivates individuals to apply for and participate in the program, and what are 
individuals' experiences during the application process? 

7. What are the program experiences of program participants and managers? 

Since 1992, IRFP has received 1,660 applications from individuals, with varied backgrounds, 
interested in pursuing their research in settings abroad. 

• IRFP is a fairly selective program, accepting just over one-third (35 percent) of applications 
overall since its inception, and it has become more competitive over time. 

• Nearly half of the proposed research projects were proposed in the life sciences (47 
percent), and about one-quarter in the physical sciences (26 percent). 

• IRFP applicants proposed research in regions that spanned the globe, although a majority 
(60 percent) identified locations in Europe. 

• Graduate advisors are reported to be supportive of their students' applications; 72 percent 
of applicants reported their advisors actively encouraged or supported their decision to 
apply. 

The IRFP applicants represent early career scientists with varied motivations for participating in the 
program, although the motivations reflect the value that these scientists and engineers place on 
foreign collaborations. 

• A large majority of applicants cited the desire to conduct research with a specific person or 
at a specific institution (87 percent), and to enhance their skills or knowledge as a 
researcher (82 percent) as the reasons for applying to IRFP. Over half also noted their desire 
to collaborate with a foreign scientist in general (60 percent) and to enhance their resume 
(57 percent). 

• In selecting a specific host country, applicants were primarily motivated by the presence of a 
host who was conducting research relevant to their own interests (87 percent), followed by 
resources at the host institution that were helpful in their research (52 percent). 

Funded fellows are hosted by research scientists in foreign institutions, who also express a 

variety of motives for participating in the program. Specifically, 

• Many foreign hosts had prior academic and professional experiences in the U.S., although 
66 percent had never hosted a postdoctoral fellow from the U.S. 

• The large majority of foreign host scientists (83 percent) reported that they agreed to host 
fellows because of the specific research projects proposed; the next most commonly 
reported rationale was an interest in creating an international environment in their research 
group (60 percent). 
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Overwhelmingly, participants of the program were satisfied with the program and found the 
fellowship valuable. 

• Former fellows and hosts offer strong endorsement for the IRFP program. All former IRFP 
fellows would recommend the program to a colleague, and the large majority of hosts (84 
percent) would recommend, or have already recommended, to others that they host an 
IRFP postdoctoral fellow from the U.S. 

• Over 90 percent of fellows were somewhat or very satisfied with the quality of research 
they were able to conduct as part of IRFP (92 percent) and the timing of the fellowship with 
respect to their career goals (97 percent). 

• Over 80 percent of fellows were satisfied with many aspects of their host, including the 
match between research interests, the host's expertise and contributions to the research, 
their inclusion in collaborations, and frequency of meetings. 

• Hosts noted that IRFP fellows compared favorably to other postdoctoral fellows with whom 
hosts had worked; 56 percent of hosts were much or somewhat more satisfied with IRFP 
fellows compared to other postdoctoral fellows, and another 28 percent were equally 
satisfied. 

• Hosts generally strongly agreed or agreed (72 and 24 percent, respectively) that their IRFP 
fellow had sufficient knowledge and expertise, and that the fellow integrated well with their 
research group (66 and 26 percent, respectively). 

The fellowships provided opportunities for collaborations and advances in research. A majority of 
fellows credited their IRFP fellowship with allowing them to make substantial advancements in their 
research, and hosts and fellows agreed they worked collaboratively on important aspects of 
research projects. 

• About two-thirds commented that their IRFP fellowship provided the chance to familiarize 
themselves with the scientific enterprise in their host site (65 percent) and to make 
substantial advancements in their research (64 percent). 

• Hosts and fellows most commonly cited collaborating on activities that involved developing 
project ideas and hypotheses, interpreting results, and planning follow-up work. 

• Hosts and fellows agreed that fellows worked mostly independently collecting data, carrying 
out simulations, and keeping records or tracking supplies and resources. 

IRFP also offers fellows a broad range of professional opportunities beyond the chance to conduct 
quality research at their host institution, including opportunities to network with colleagues from 
other than their host institution, attend lectures in their field, and give talks at their host institution. 

• Other professional activities featured prominently in the experiences of many fellows. 
Specifically, IRFP provided fellows with the opportunity to network with colleagues from 
institutions outside their host institution (90 percent), attend lectures in their field (89 
percent), visit other institutions (89 percent), attend conferences (84 percent), and give 
talks at their institution (71 percent). 
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• Fellows also experienced a variety of cultural activities including sightseeing (96 percent) 
outdoor activities that explore geography (95 percent), museums (89 percent), and festivals 
or holidays (85 percent). 

Most participants did not cite specific challenges, but those that were mentioned illustrate areas 
that might deserve attention of future fellows or hosts. 

• Forty percent of fellows reported no noteworthy difficulties during their fellowship. The 
most common challenges were related to logistics (32 percent) and language difficulties (20 
percent), followed by inadequate access to space or resources (16 percent), and not enough 
guidance from the host researcher or research group (15 percent). 

• Only one-third of hosts reported any challenges experienced in hosting an IRFP fellow. 
Specific challenges in hosting an IRFP fellow were identified by less than 10 percent of hosts, 
including noting that the fellow did not devote enough time to the research collaboration 
(10 percent), and the fellow worked too independently or did not work well as a 
collaborator (8 percent). 


Evaluation Approach 


The evaluation employed quasi-experimental impact analyses to compare the outcomes of fellows 
to those of unfunded applicants, using pre-award characteristics of applicants to mitigate the 
potential threat of selection bias. To reduce the risks associated with selection bias, the study 
incorporated propensity score analysis (PSA) to construct groups of awardees and non-awardees 
that were statistically similar across a number of pre-existing characteristics. A secondary set of 
comparative analyses between IRFP applicants (and fellows) and a nationally representative sample 
of STEM doctorates from the Survey of Doctoral Recipients (SDR) was used to situate the outcomes 
of IRFP program participants and applicants within the national S&E context. The evaluation also 
used descriptive analyses to explore the pre-award international research experiences and other 
characteristics of IRFP applicants and host scientists; to understand what motivated STEM doctoral 
recipients to apply for an IRFP postdoc and what led IRFP host scientists to collaborate with an IRFP 
fellow; and to describe the experiences of IRFP participants (i.e., fellows and hosts), both during and 
after the period of the postdoctoral fellowship. 

Data for the evaluation were drawn from extant sources, and surveys were administered to collect 
information from program applicants (both those who received IRFP fellowships and those who did 
not) and IRFP foreign hosts. Extant data came from NSF's administrative records on applicants and 
from the Survey of Doctoral Recipients (SDR). The core data for the evaluation were gathered 
through online surveys completed— from January through March, 2011— by IRFP applicants and 
hosts. 

The target populations for the study included all individuals who had applied to the IRFP program 
from its inception in 1992 through 2009, as well as the research scientists who served as foreign 
hosts during this period. Responses were received from 457 IRFP fellows and 582 unfunded 
applicants; responses rates were 81 percent and 55 percent, respectively, and estimates were 
adjusted for non-response. Surveys were received from 328 hosts, for a 61 percent response rate. 
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1 Introduction 


Effective international S&E partnerships advance the S&E enterprise and energize U.S. innovation 
and economic competitiveness, but they also have great potential to improve relations among 
countries and regions and to build greater S&E capacity around the world . 6 

Over a decade ago, the National Science Board (NSB) highlighted the importance of international 
collaboration in its call for increased government commitment to promoting international science 
and engineering (S&E) research and education. 7 The NSB also identified the National Science 
Foundation (NSF) as having an important leadership role in international S&E research and 
education activities. 8 A specific area of focus for NSF was promoting "...increased participation in 
international S&E activities by younger U.S. scientists and engineers from diverse backgrounds, 
especially those in the early stage of their careers, in order to develop an internationally competitive 
and globally-engaged S&E workforce." 9 

Within NSF, the Office of International Science and Engineering (OISE) serves as a primary hub for 
international research opportunities for U.S. scientists and engineers. OISE's International Research 
Fellowship Program (IRFP) provides international fellowships to early career scientists to further 
NSF's commitment to support the active engagement of early career STEM researchers in 
international collaborations. 

NSF staff are familiar with successful IRFP projects, and have shared highlights of individual IRFP 
projects to disseminate program successes more broadly to the S&E community. (See Appendix A 
for NSF-generated descriptions of some such highlighted IRFP projects). However, rather than 
relying on anecdotal evidence of program success for making programmatic decisions, NSF sought to 
systematically gather data on the program through an external evaluation. In August 2009, NSF 
contracted with Abt Associates to conduct an evaluation of the IRFP, to gather evidence about 
whether the IRFP program has achieved its goal of furthering the collaborative activities and 
international partnerships of early career STEM researchers. This report presents the findings from 
Abt Associates' evaluation, focusing on program applicants over the 1992-2009 time period as well 
as their foreign hosts. 

The evaluation found evidence that the IRFP program is meeting its goals, which are to: 

• Introduce early career scientists and engineers to opportunities for international research 
collaboration; 

• Build research capacity and global perspective of participants; and 

• Forge long-term relationships between U.S. and foreign S&E researchers. 


NSB. 2008. 

7 NSB. 2001. 

8 NSB. 2000. 

9 NSB. 2000, p. 10. 
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This report presents the findings from this evaluation. Below, the IRFP program is described and 
situated within recent trends in international scientific research. The report then describes the 
methodology used to conduct the evaluation (Chapter 2); characteristics of IRFP applicants and their 
motivations for participation, and the host researchers' motivations for participating in the program 
(Chapter 3); participants' experiences in and perceptions of the program (Chapter 4); outcomes of 
the program (Chapter 5); and conclusions and implications of the findings of the evaluation. The 
appendices included with this report provide additional details: Appendix A presents specific project 
highlights that NSF has developed, Appendices B through D describe methodological details, 
Appendix E contains the survey instruments, and Appendix F contains a comparison of IRFP 
applicants to national data on career outcomes. 


1.1 IRFP 


Postdoctoral positions are playing a role in the research development of more PhD graduates. For 
example, according to data from the Survey of Doctorate Recipients, the numbers of S&E PhD 
graduates reported having had a postdoctoral position has risen from 31 percent among PhD 
holders who graduated prior to 1972, to 46 percent among those who graduated from 2002 to 
2005. 10 Among NSF's postdoctoral programs, IRFP is unique in its emphasis on providing 
postdoctoral fellows with international research experiences. 11 IRFP aligns closely with the NSB's call 
for NSF to support the international science engagement of scientists and engineers. 

The program logic model for IRFP (Exhibit 1.1) identifies the links between program processes and 
outcomes, and helped ground the evaluation in the program's theory. The IRFP program goal is to 
introduce early career scientists to international collaborative research opportunities that further 
their research capacity and global perspective, and that forge long-term relationships between U.S. 
and foreign scientists, engineers, and technologists. The fellows' selections of research topics and 
appropriate foreign hosts are expected to lead to joint fellow-host publications, better 
understanding by the fellows of the host country's cultural and academic conditions, and ideally, to 
continued research collaboration initiated through and persisting beyond the duration of the 
fellowship experience. In addition to the internal components of the logic model, factors that are 
outside of the participants' control might shape the program; some of these factors are listed in the 
model. For example, funding levels allocated to the program each year will affect the number of 
fellows receiving support. Safety and economic conditions in foreign countries, as well as the level of 
diplomatic engagement between the United States and other countries, can also influence the 
fellows' selection of the hosts. 


NSB. 2010a. Science and Engineering Indicators 2010. NBS-10-01. National Science Foundation: Arlington, 
VA. 

A list and descriptions of NSF postdoctoral opportunities can be found at 
http://www.nsf. gov/funding/education. jsp?fund_type=3 
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Exhibit 1.1: IRFP Program Logic Model 
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IRFP PROGRAM GOALS 

• To introduce early career scientists and engineers to international 
collaborative research opportunities 

• To further their research capacity and global perspective 

• To forge long-term relationships between U.S. and foreign scientists, 
engineers, and technologists 


IRFP PROGRAM ACTIVITIES 

NSF 

• Development of program solicitation 

• Selection of awardees 

• Program monitoring 

Fellows 

• Development of proposal and selection of international host institution 

• Conducting research projects at host institution 

• Submission of annual and final reports 

Foreign host institutions 

• Provision of facilities, research environment, and mentoring 


IRFP PROGRAM OUTPUTS 

• Annual and final reports 

• Joint publications, patents, funding applications 

• Tools, data sets, samples, instruments 

• Research projects suitable for continued partnership 


IRFP PROGRAM OUTCOMES 

• Participant familiarity with international science and engineering enterprise 

• Familiarity with host country's research enterprise, language, and customs 

• Career choices involving opportunities for international collaboration 

• Advancement of U.S. and foreign scientists' research agendas 

• Development of ties between U.S. institutions and foreign hosts 

• Awareness of opportunities for international collaboration in participant 
institutions 


IRFP PROGRAM IMPACTS 

• Emergence of a cadre of scientists and engineers who will play a 
leadership role in forging international collaborations 

• Development of collaborations that are of great value to the nation 

• Maintenance of U.S. leadership in science and technology 

• Research capacity building in foreign institutions 
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Established in 1992, IRFP provides financial support to postdoctoral scientists for a research 
experience abroad lasting from 9 to 24 months. 12 There is no restriction on the geographical location 
of the hosting institution; in 2008, for example, analyses of programs records show that 31 fellows 
resided in 18 countries. 

These goals have been fairly stable throughout the program; they have undergone only two small 
changes since its inception. The wording in the first part of the goal "to introduce scientists and 
engineers in the early stages of their careers to opportunities abroad" has remained unchanged. The 
second part of the goal was refined, first in 2003, from "thereby furthering NSF's goal of establishing 
productive, mutually beneficial relationships between U.S. and foreign science and engineering 
communities" to "thereby furthering NSF's goal of creating a diverse, competitive, and globally- 
engaged U.S. workforce of scientists, engineers, and technologists, and well-prepared citizens" and a 
second time in 2007 from "thereby furthering NSF's goal of creating a diverse, competitive, and 
globally-engaged U.S. workforce of scientists, engineers, and technologists, and well-prepared 
citizens" to "thereby furthering their research capacity and global perspective and forging long-term 
relationships with scientists, technologists, and engineers abroad." 

The program has changed in scope since its inception in 1992. For instance, program records show 
that the number of annual awards has gradually increased from 20 to 35, and the program's funding 
amount increased from approximately $1 million in 2000 to almost $3.5 million in 2009. 
Consequently, the average IRFP award amount gradually increased from $50,000 in 1992 to 
$150,000 in 2009. Applications have always been accepted once every year, though application 
deadline dates have ranged from September through November. The fellowship period has 
remained 9 to 24 months. 


1.2 Globalization of Science and Engineering and International 
Collaboration 


International research partnerships are increasingly important to advancing knowledge and 
discoveries in science, technology, engineering, and mathematics (STEM) fields and for addressing 
problems of a global nature. A decade ago, the NSB noted that "international boundaries have 
become considerably less important in structuring the conduct of research and development" in 
S&E fields. 13 Ten years later, the globalization of S&E research and education continues. The NSB 
determined that the importance of these trends, as well as the associated opportunities and 
challenges for the U.S., warranted a publication that highlighted the globalization of science and 
engineering research. 14 

Further, the potential of science policy and science diplomacy to meet international challenges was 
the focus of a workshop convened by the Committee on Global Science Policy and Science 


NSF. 2006a. 

NSB. 2001, p.12 

NSB. 2010b. Globalization of science and engineering research: A companion to science and engineering 
indicators 2010. NBS-10-3. Arlington, VA: National Science Foundation. 
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Diplomacy of the National Research Council. 15 Participants discussed the importance of international 
science engagement and global science cooperation; they noted the importance of providing 
opportunities and incentives for U.S. researchers to engage in science in an international arena. 16 

While recent data emphasize that the U.S. is still a major global force in S&E education, research, 
and investment, other countries are increasing their competitiveness in these areas. The 2010 
Science and Engineering Indicators report illustrated that the S&E human capacity of foreign 
countries continues to grow. 17 For instance, other countries are increasingly competing with the U.S. 
for foreign students, and top student destinations include countries such as the United Kingdom, 
Germany, and France; in fact, the proportion of foreign students in the U.S. decreased from 25 
percent in 2000 to 20 percent in 2006. 18 Additionally, other countries are producing more S&E 
graduates than the United States; of the more than 4 million first university S&E degrees awarded 
worldwide in 2006, 21 percent were earned by students in China, 19 percent were awarded to 
students from the European Union, and only 11 percent of these degrees were earned by students 
in the United States. 19 

Data also show that there has been growth in foreign nations' S&E workforces that can compete 
internationally. For instance, between 1997 and 2007 the estimated number of S&E researchers in 
the United States grew by 40 percent to reach approximately 1.4 million. However, over the same 
time period, the number of S&E researchers in China grew by 173 percent to also reach 1.4 million. 20 
Additionally, while the U.S. remains the leader in S&E research and development expenditures 
(accounting for 33 percent of total spending), countries such as China have begun to invest large 
amounts of money into their own research and development (R&D) expenditures (averaging 19 
percent annually over the past decade), and the U.S. R&D to gross domestic product (GDP) ratio 
now ranks eighth among economies tracked by the OECD. 21 Finally, while worldwide S&E research 
article output has grown at an average annual rate of 2.5 percent between 1995 and 2007, the U.S. 
output has only grown by 0.7 percent. Other countries' rapid growth in S&E fields means they are 
likely to train S&E researchers who can compete on a global level, as well as attract skilled workers 
who might have otherwise chosen to work in the U.S. 

In light of these recent trends, it has become increasingly important for the U.S. to assume a 
leadership role in the formation of international research partnerships. It is important for U.S. 
scientists to fully engage in international collaboration to maintain a prominent role in the global 
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16 
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National Research Council, Committee on Global Science Policy and Science Diplomacy. 2011. U.S. and 
International Perspectives on Global Science Policy and Science Diplomacy: Report of a Workshop. 
Washington, DC: National Academies Press. 

NRC. 2011. 
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research community and gain insight into international research advances . 22 By assuming an active 
role in international collaborations, the U.S. can expect to reap benefits that might not otherwise be 
realized. For instance, Luo et al. found that when researchers from the U.S. and the U.K. engaged in 
collaborations, the impact of their resulting research significantly increased (as measured by citation 
rates), especially for U.S. corresponding authors . 23 Additional benefits of international collaboration 
may include increased access to physical resources and funding; additional opportunities to benefit 
from the expertise of collaborators; and access to populations, records, historical materials, and 
circumstances that provide "natural experiments ." 24 Finally, allowing for international collaboration 
also serves as a way to facilitate the expansion of U.S. markets and to promote opportunities for 
international economic exchange . 25 

The promotion of international collaboration also has impacts beyond the United States. As experts 
note, research and development in S&E fields can be costly, and it is increasingly necessary for 
countries to "transcend national boundaries in order to be able to fund projects ." 26 Additionally, 
important problems such as terrorist threats, climate change, and disease outbreaks exist, and 
therefore must be solved, on a global scale. By establishing international networks of scientists, 
resources can be shared and ideas can be developed, tested, and implemented across traditional 
boundaries . 27 

Finally, these partnerships can serve as an important tool in broader international diplomacy efforts. 
As the National Science Board explains, "science and engineering partnerships can strengthen 
international relationships and. ..promote basic scientific values such as accountability, meritocracy, 
transparency, and objectivity ." 28 Policymakers also note that the inclusion of developing nations in 
these collaborative efforts can promote self-sufficiency and encourage international participation in 
a variety of areas beyond S&E research . 29 


1.3 Programs to Promote International Research and Collaboration 


Funding for international science and engineering partnerships is concentrated in a handful of 
federal agencies: the Department of Defense (DOD), the National Institutes of Health (NIH), and 


NSB 2001. 


Luo, J., Flynn, J.M., Solnick, R.E., Ecklund, E.H., & Matthews, K.R.W. 2011. International stem cell 
collaboration: How disparate policies between the United States and the United Kingdom impact 
research. PLoS ONE, 6(3), el7684 

24 Goodnow, J. 2006. The benefits of cross-cultural collaboration. In International Collaborations in 
Behavioral and Social Sciences Research: Report of a Workshop (pp. 47-63). Washington, D.C.: National 
Academies. Retrieved from www.nap.edu/openbook.php?record_id=12053&page=47 

25 NSB. 2001. 

26 NSB. 2001. 

27 NSB. 2008. 

28 NSB. 2008. 

29 NSB. 2008 
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NSF. 30 Among these agencies, NSF is unique in its emphasis on basic science and engineering. Within 
NSF, a large number of programs that support international partnerships primarily reside in OISE. As 
part of its activities, OISE administers the International Research Fellowship Program (IRFP), which 
provides support to postdoctoral scientists (generally a year or two after the receipt of a doctoral 
degree), for a 9- to 24-month research experience abroad. Another program that aims to promote 
long-term international collaborative research (also sponsored by NSF) is the Partnerships for 
International Research and Education (PIRE) program. This program funds proposals from all areas 
of NSF-supported science in an effort to: enhance research excellence through international 
partnerships; promote educational excellence via international collaborations; and strengthen U.S. 
capacity for international engagement. There is no specific budget limit for any PIRE award. 31 NSF 
also supports programs that facilitate short-term international collaborations; these include the East 
Asia and Pacific Summer Institutes (EAPSI), which provides support to U.S. graduate students to 
spend 8-10 weeks over the summer conducting research in one of seven countries in East Asia and 
the Pacific region; 32 the Pan-American Advanced Studies Institutes (PASI) program (jointly sponsored 
with the Department of Energy), which funds students or junior researchers to participate in short 
courses designed to disseminate knowledge and stimulate training and cooperation among 
researchers from countries in the Americas; 33 and the Catalyzing New International Collaborations 
program, which provides funding for short-term activities such as planning visits, workshops, initial 
data-gathering activities, and the development of research networks designed to develop 
international collaborations. 34 

Of course, other U.S. agencies and organizations recognize the value of international collaborations, 
and have implemented programs to facilitate the international exchange of S&E research. For 
instance, the NIH includes the Fogarty International Center, which funds programs to support global 
health research conducted through international partnerships, builds partnerships between 
research institutions in the U.S. and abroad, and trains the next generation of scientists to address 
global health issues. 35 Another example of a U.S. government-funded effort to promote 
international collaboration is the International Fulbright Science and Technology Award, sponsored 
by the U.S. Department of State, which allow international participants to complete a PhD in the 


NSB. 2008. 

31 NSF. 2006b. Partnerships for International Research and Education (PIRE) (NSF 09-505). Retrieved from 
http://www.nsf.gov/pubs/2009/nsf09505/nsf09505.pdf 

32 NSF. 2010. East Asia and Pacific Summer Institutes (NSF 10-591) Retrieved from 
http://www.nsf. gov/funding/pgm_summ.jsp?pims_id=5284 

33 NSF. 2006c. Pan-American Advanced Studies Institutes Program (PASI) (NSF 10-517). Retrieved 

from http://www.nsf.gov/pubs/2010/nsfl0517/nsfl0517.pdf 

34 NSF. 2006d. Catalyzing New International Collaborations (NSF 11-508). Retrieved from 
http://www.nsf.gov/pubs/2011/nsfll508/nsfll508.pdf 

35 The John E. Fogarty International Center: U.S. Department of Health and Fluman Services, National 
Institutes of Health, (n.d.). Fogarty at 40: Advancing Science for Global Health. Retrieved from 
http://www.fic.nih.gov/news/publications/fogarty_40th_brochure.pdf 
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United States. The Science and Technology Award program has awarded grants to 155 students 
hailing from 69 counties worldwide. 36 

Additionally, foreign governments have also established programs to facilitate collaboration 
between U.S. and international S&E researchers. For instance, the Research Internships in Science 
and Engineering (RISE) program, administered by the German Academic Exchange Service, provides 
funding for undergraduates to engage in a summer research internship. Participants are matched 
with German PhD students, who serve as their mentors. 37 Another example, the Human Science 
Frontier Program (HFSP), provides Young Investigators' Grants, which allow researchers who have 
completed one or two periods of postdoctoral training to direct their own research project (HSFP 
Research Grants 2011). 38 A final example, the Partner University Fund, was designed to foster 
educational research networks between the United States and France. The fund supports 
approximately 40 partnerships by administering three-year grants that can reach up to $80,000 
annually. 39 


1.4 Outcomes of International Partnership Programs 


Limited research is available on the effectiveness of programs designed to promote international 
S&E collaboration. However, evaluations of programs similar to IRFP have found that students who 
participated in these programs reported gaining new knowledge and skills during their international 
collaboration. Many also reported that they would be likely to engage in international collaborations 
in the future, perhaps as a result of their international research experience. For instance, an 
evaluation of NSF's International Research Experience Program, a program for undergraduate and 
graduate students that ended in 2008, found that students who engaged in an international 
research experience reported gaining technical, communication, and language skills; developed an 
appreciation for cultural differences; and felt that their research experience would "create 
opportunities for future international collaboration." 40 An evaluation of the Research Internships in 
Science and Engineering (RISE) program reached similar conclusions. The RISE program provides 
funding for undergraduate students to complete summer internships in German higher education 
institutions. Researchers found that the majority of RISE student participants developed a greater 
understanding of German culture, the vast majority reported an increased desire to travel abroad, 
and about 1 in 4 alumni reported returning to Germany at some point in the future, either to visit or 


The International Fulbright Science & Technology Award. Retrieved from 
http://scienceandtech.fulbrightonline.org/more-about-sat 

37 Institute of International Education. 2009. Evaluating the DAAD's Research Internships in Science and 
Engineering (RISE) Program: A Final Report 

3S Human Frontier Science Program. 2010. Guidelines for Applicants: Award Year 2011. Retrieved from 
http://www.hfsp.org/sites/www.hfsp.org/files/webfm/Grants/LI2012%20Guidelines.pdf 

39 Partner University Fund. 2011. A New Generation of Transatlantic University Partnerships. Retrieved from 
http://facecouncil.org/puf/wp-content/uploads/2009/10/BrochurePUF20111.pdf 

40 Spencer, D. 2008. International research experience program: International research opportunities for 
students at NSF science and technology centers. Retrieved from 
http://66.116.177.96/IREP%20Evaluation%20Report.pdf 
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to pursue work/study opportunities . 41 Finally, an evaluation of NSF's International Research and 
Education in Engineering (IREE) pilot program also found that program participants, especially 
graduate and postdoctoral students, reported acquiring new research capabilities, and that they 
planned to continue collaborations with their international counterparts at the end of their program 
experience . 42 


1 .5 Purpose of This Study 


The importance of international collaboration cannot be overstated. Researchers and policymakers 
agree that while the United States remains a leader in S&E research and development, the nation 
cannot maintain this position without engaging in international collaborations. Promoting 
international engagement at all levels is crucial to fostering successful research partnerships and 
developing the next generation of S&E researchers. Providing early career researchers with an 
opportunity to engage in an international research experience may help them improve their own 
research capabilities and pursue future collaborations with international colleagues. This, in turn, 
could lead U.S. researchers to reap benefits such as increased visibility in the research community, 
access to more substantial funding and resources, and the opportunity to benefit from the expertise 
of international peers. 

To align with the program goals, the evaluation paid particular attention to the opportunities for 
international research collaborations, the experiences leading to developing research capacity and 
global perspectives, and the relationships between U.S .and foreign researchers. Thus, the 
evaluation was designed to: describe the experiences with the program of IRFP participants; explore 
the educational and career outcomes of IRFP fellows and how they compare to those of unfunded 
applicants and to national doctoral degree recipients; and investigate the effects of the programs 
beyond the direct participants to other scientists and on institutions. 


Institute of International Education, 2009. 

Flattau, P.E., Lai, B., Laskey, A., & Ford, J. J. 2009. Portfolio Evaluation of the National Science Foundation's 
Grants Program on "International Research and Education in Engineering" (IREE). Washington, DC: 
Institute for Defense Analyses, Science & Technology Policy Institute. 


Abt Associates Inc. 


1. Introduction 


15 




Evaluation of International Research Fellowship Program: Final Report 


2 Methodology 


It was important for this study to both measure individuals' experiences with the program and to 
investigate the effects of the program on participants, particularly with respect to their international 
research collaborations. As detailed below, the study used a mixed methods design, incorporating 
extant and primary data sources, to answer the specific research questions. This evaluation included 
both descriptive and comparative analyses. The descriptive analysis provided summary information 
about the characteristics, experiences, and perceptions of individuals involved with the program, as 
well as descriptive comparisons of differences between groups or points in time. The main 
comparative analyses were designed to investigate the effects of the program on participants, and 
specifically attribute any observed differences to participation in IRFP. Thus, quasi-experimental 
impact analyses compared the outcomes of fellows to those of unfunded applicants, using pre- 
award characteristics of applicants to mitigate the potential threat of selection bias. A secondary set 
of comparative analyses between IRFP applicants (and fellows) and a nationally representative 
sample of STEM doctorates from the Survey of Doctoral Recipients (SDR) was used to situate the 
outcomes of IRFP program participants and applicants within the national S&E context. 


2.1 Research Questions 


This study was designed to evaluate the extent to which NSF's IRFP program contributes to the 
engagement of postdoctoral S&E researchers in international research collaborations. Specifically, 
the evaluation investigated the extent to which NSF is achieving program goals by answering the 
following research questions: 

1. What are the characteristics of people who apply for and participate in the IRFP program? 

2. What motivates individuals to apply for and participate in the program, and what are 
individuals' experiences during the application process? 

3. What are the program experiences of program participants and managers? 

4. What are the perceived outcomes of program participation? 

5. Does the extent to which former fellows engage in international collaborations differ from 
those of unfunded applicants? 

6. Do fellows' post-award career activities and job characteristics differ from unfunded 
applicants? 

7. Do the outcomes of program participation extend beyond the direct participants? 


2.2 Data Sources 


Data for the evaluation were drawn from extant sources, and surveys were administered to collect 
information from program applicants (both those who received IRFP fellowships and those who did 
not) and IRFP foreign hosts. 
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2.2.1 Extant Data 

Extant data came from NSF's administrative records on applicants and from the Survey of Doctoral 
Recipients (SDR). NSF's administrative records on applicants were used to construct the study 
sample. Existing national data from the SDR were used to contextualize IRFP applicants' and 
awardees' experiences and outcomes relative to a nationally representative sample of STEM 
doctorates who have earned a doctorate in a science, engineering, or health field. 

2.2.2 Primary Data 

The core data for the evaluation were gathered through online surveys completed— from January to 
March, 2011— by IRFP applicants and hosts. The surveys gathered information about experiences 
prior to, during, and after the IRFP program. The IRFP applicant survey had some modules that were 
specific to either unfunded applicants or IRFP fellows. Copies of this survey and the IRFP foreign host 
survey are included in Appendix E. 


2.3 Study Sample 


The target populations for the study included all individuals who had applied to the IRFP program 
from its inception in 1992 through 2009, as well as the research scientists who served as foreign 
hosts during this period. These samples are described below. 

IRFP fellows and unfunded applicants. All individuals who applied to the IRFP program between 
1992 and 2009 were included in the IRFP study. Individuals may have applied more than once to the 
IRFP program, but for the purposes of this study, they were assigned to a single award status as 
follows: applicants who ever received a fellowship were considered awardees; applicants who never 
received a fellowship were considered non-awardees. If individuals received more than one award, 
they were included in the study for their most recent award. Exhibit 2.1 shows the sample size and 
response rate for the applicant survey. 

The universe of study-eligible applicants included 581 awardees and 1079 non-awardees (n=l,660 
applicants). Thirty-two applicants (17 awardees and 15 non-awardees) were not eligible for the 
study and were excluded from the sampling frame. 43 Eliminating these applicants reduced the 
eligible sample to 1,628 applicants (564 awardees and 1,064 non-awardees). The overall response 
rate for the applicant survey was 64 percent. Response rates for the two groups were 81 percent 
(n=457) for awardees and 55 percent (n=582) for non-awardees. 


Individuals were excluded for one of the following reasons: individual had participated in a pilot test of 
the applicant survey; individual was deceased; individual was awarded the IRFP fellowship but declined 
the award; potential conflict of interest— individual works for NSF; individual did not recall applying for an 
IRFP fellowship; or inspection of data showed that individual was not eligible to have applied for IRFP or 
to participate in the study. 
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Exhibit 2.1: Final Sample Size and Response Rates for the IRFP Applicants Survey 



Overall 

Awardees 

Non-Awardees 

a. Target sample 

1,660 

581 

1,079 

b. Final survey sample a 

1,628 

564 

1,064 

c. Number of completed & 

1,050 

460 

590 

partially b completed 




surveys 




d. Number of completed 

1,039 

457 

582 


surveys 


Response Rates 




e. response Rate (d/b) 

64% 

81% 

55% 


3 32 individuals were determined to be ineligible for the IRFP study. 

b 11 respondents were classified as "partial" responders because they had completed less than 15 percent of the 
questions on the survey. 


IRFP hosts. All hosts of eligible awardees who were identified in NSF records were included in the 
IRFP study. Flosts for whom records were not available were not included. The universe included 
557 study-eligible hosts. Twenty-one hosts were not eligible for the study and were excluded from 
the sampling frame. 44 Eliminating these hosts reduced the eligible sample to 536 hosts. The overall 
response rate for the host survey was 61 percent (n=328). 


Exhibit 2.2: Final Sample Size and Response Rates for the IRFP Host Survey 



Overall 9 

a. Target sample 

557 

b. Final survey sample a 

536 

c. Number of completed & partially b completed surveys 

335 

d. Number of completed surveys 

328 

Response Rates 


e. Response rate (d/b) 

61% 


a 21 individuals were determined ineligible for the IRFP study. 

b 7 respondents were classified as "partial" responders because they had completed less than 5 percent of the 
questions on the survey. 


Non-Response 


An initial step in the analysis was to explore the consequences of survey non-response, which could 
lead to bias if the former fellows or unfunded applicants who did not participate in the study would 
have given systematically different responses to the survey than the individuals who did participate 
in the survey. Two types of non-response were investigated: unit non-response, that is, instances 
where no survey was completed; and item non-response, where individual items of an otherwise 
completed survey were missing. 


44 Individuals were excluded for one of the following reasons: individual was deceased; individual was 
hosting an applicant who was awarded the IRFP fellowship but had declined the award; individual 
reported not hosting an IRFP fellow; or inspection of data showed that individual was not eligible to 
participate in the study. Although it is likely that these data were erroneous, it was not possible to verify 
or correct the information provided. 
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2.4.1 Unit Non-Response 

The overall response rate for both unfunded applicants and fellows was less than 80 percent. To 
address unit non-response for the applicant survey, information from NSF program records was 
used to estimate the probability that a person responded to the survey, as a function of baseline 
characteristics that were available (e.g. proposal score, cohort year, gender). These probabilities 
were used to create weights that were then used to adjust estimates to alleviate the potential bias 45 
due to non-response. This method is described in more detail in Appendix B. 

The overall response rate for the host survey was less than 80 percent; however, absent available 
extant data on hosts, it was not possible to conduct a non-response bias study. Hence findings refer 
only to the survey respondents, and not to all IRFP hosts. 

2.4.2 Item Non-Response 

Item non-response refers to information missing on one or more specific items on an otherwise 
completed survey. Since the amount of missing data on an individual item was modest (less than 5 
percent) across all returned surveys, it was assumed that missing data on an item are missing at 
random; information on missing data is presented in all exhibits in this report. Where appropriate, 
the study imputed missing covariate values. Outcome variables were not imputed (for more details 
on specific steps taken, see Appendix B). 


2.5 Analyses 


A series of analyses were conducted to answer the descriptive, comparative and impact study 
questions, as described below. The findings from these analyses are described in the next three 
chapters. 

2.5.1 Descriptive Analyses 

Most of the research questions were addressed through the use of simple descriptive statistics such 
as means and percentages, as well as cross-tabulations to illustrate patterns of responses for 
groups, or the distribution across subgroups of interest. 

In general, categorical variables are summarized in terms of the percentage of respondents who 
indicated a particular response, and continuous outcomes are presented in terms of means. All 
exhibits include information on the total number of respondents on which percentages or means 
were computed as well as information on item non-response. Estimates are adjusted using weights 
to account for unit non-response (to mitigate any potential bias) so that parameter estimates are 
representative of the IRFP program as a whole. 


Note that a large non-response rate does not necessarily create bias. For example, if the non-respondents 
were similar across the awardees and non-awardees, then the impact estimate would not be biased 
necessarily; rather, any effect of the program could not be generalized to the non-respondents (i.e., it 
would create an external validity problem but not necessarily an internal validity issue). 
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Responses to survey items that include an "other, please specify" with the option for entering text 
were coded by the study team for classification into one of the existing response options, where 
possible. 

The qualitative data from open-ended survey questions, for example survey items that asked 
respondents to describe their individual experiences and perceptions, were examined for common 
themes and standard coding techniques were applied, where appropriate. These open-ended 
responses are described in the findings sections and used to illustrate specific experiences with the 
program. These responses provide detailed examples of individual experiences, although one cannot 
generalize from these data to all respondents as not all respondents necessarily responded to each 
open-ended question. Thus, what one respondent described could reflect an idiosyncratic 
perception or a view shared by others who left an open-ended item blank, or chose to emphasize 
something else in their comments. Moreover, those who chose to write an open-ended response 
may have had systematically different views than those who chose to leave such items blank. 

2.5.2 Impact Analyses 

The evaluation was designed to answer the following research questions about the impact of the 
IRFP award on participants: 

• Do fellows' international collaborations and other international activities differ from those 
of unfunded applicants in frequency, length, and type of activities? 

• What are the fellows' post-award career activities and job characteristics? How do these 
compare to unfunded applicants and the national samples? 

The goal of the impact component of the evaluation was to estimate the effect of IFRP on its 
participants. If a program brings about changes in its participants, then these individuals should have 
different outcomes, post-participation, than they would have had in the absence of program 
participation. Although a random assignment study would have allowed a more rigorous test of the 
causal impact of the IRFP award on its recipients, this design was not feasible (nor, perhaps, 
desirable) since awardees had already been selected based on the merits of their IRFP proposal. 
Instead, a quasi-experimental design was used to compare outcomes for IRFP awardees and non- 
awardees. The primary threat to the validity of a quasi-experiment comes from selection bias, 
namely, the possibility that pre-existing differences between awardees and non-awardees, rather 
than the IRFP award itself, are responsible for observed differences in outcomes between the two 
groups. 

To reduce the risks associated with selection bias, the study incorporated propensity score analysis 
(PSA) to construct groups of awardees and non-awardees that were statistically similar across a 
number of pre-existing characteristics (e.g., gender, prior international experience, etc.). 46 


46 PSA is a common quasi-experimental design approach that has been shown to produce unbiased 
estimates of program effects. See for example Rosenbaum, P., & Rubin, D. 1984. Reducing bias in 
observational studies using subclassification on the propensity score. Journal of the American Statistical 
Association, 79(387): 516-524; Heckman, J., Ichimura, H., Smith, J., & Todd, P., Characterizing selection 
bias using experimental data, available from 

http://www.irp.wisc.edU/initiatives/trainedu/igrfp/readings04/Heckman_Characterizing_selection_bias.p 
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Subsequent impact models incorporated the results of the PSA. These methods are described in 
more detail in Appendix C. 

For each outcome, the impact of IRFP was estimated for each propensity stratum (controlling for 
number of years since PhD degree, underrepresented minority status, and gender, and also where 
applicable number of pre-award publications and field of study); then, the overall treatment effect 
was calculated by taking an average of the estimated treatment effects weighted by the number of 
treated observations (i.e., the number of awardees) within each stratum. Exhibits display the 
adjusted (estimated) means for awardees and non-awardees, the estimated impact, the standard 
error, and the p-value. For model specifications and standard error calculations, see Appendix C. 

It is important to note that the findings described here result from a quasi-experimental analysis 
that incorporates statistical controls. Although the propensity score analysis is used to control for 
pre-existing characteristics along which awardees and non-awardees may have differed, there is 
always a chance that some unmeasured preexisting characteristic, rather than the IRFP award itself, 
could be responsible for any difference in outcomes between these two groups. 

2.5.3 Comparative Benchmarking 

The 2006 and 2008 Survey of Doctoral Recipients (SDR) was utilized in this study to compare the 
IRFP fellows and applicants to a nationally representative sample of science, engineering and health 
(SEH) doctoral degree recipients on key employment, postdoctoral appointment, and international 
collaboration variables. For this study, the primary comparison group for IRFP fellows is a 
propensity-score matched sample of unfunded IRFP applicants. The SDR respondents were used as a 
secondary comparison group to assess how fellows' and all applicants' outcome indicators compare 
to national averages. SDR indicators most relevant to the study include employment sector and 
current position, and the nature and extent of collaboration with foreign researchers. The methods 
used for these analyses are described in more detail in Appendix D. 


df; Cook, T., Shadish, W. and Wong, V. 2008. Three conditions under which experiments and observational 
studies produce comparable causal estimates: New findings from within-study comparisons, Journal of 
Policy Analysis and Management, 27(4), 724-750. 
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3 Characteristics of Program Applicants and Participants 


This chapter describes the characteristics of IRFP applicants, including background characteristics, 
pre-application international experiences, and motivations for seeking an IRFP award. It also 
describes whether and how such characteristics differ for unfunded applicants and awarded IRFP 
fellows. Findings on the applicants' characteristics are drawn from the 1,039 IRFP applicants who 
completed the applicant survey (457 IRFP fellows, 582 unfunded applicants); estimates have been 
weighted to adjust for non-response as described above in Chapter 2: Methodology. In addition, this 
chapter presents information about the IRFP host scientists' reasons for participating in the 
program, drawing from the 328 completed IRFP host surveys. 

Specifically, this chapter answers the following questions: 

• What are the characteristics of people who apply for and participate in the IRFP program? 

• What motivates individuals to apply for and participate in the program? 


3.1 Key Findings 


• IRFP is a fairly selective program, accepting just over one-third (35 percent) of applications 
overall since its inception, and it has become more competitive over time. 

• Nearly half of the proposed research projects were proposed in the life sciences (47 
percent), and about one-quarter in the physical sciences (26 percent). 

• IRFP applicants proposed research in regions that spanned the globe, although a majority 
(60 percent) identified locations in Europe. 

• Graduate advisors are reported to be supportive of their students' applications; 72 percent 
of applicants reported their advisors actively encouraged or supported their decision to 
apply. 

• A large majority of applicants cited the desire to conduct research with a specific person or 
at a specific institution (87 percent), and to enhance their skills or knowledge as a 
researcher (82 percent) as the reasons for applying to IRFP. Over half also noted their desire 
to collaborate with a foreign scientist in general (60 percent) and to enhance their resume 
(57 percent). 

• In selecting a specific host country, applicants were primarily motivated by the presence of a 
host who was conducting research relevant to their own interests (87 percent), followed by 
resources at the host institution that were helpful in their research (52 percent). 

• Many foreign hosts had prior academic and professional experiences in the U.S., although 
66 percent had never hosted a postdoctoral fellow from the U.S. 

• The large majority of foreign host scientists (83 percent) reported that they agreed to host 
fellows because of the specific research projects proposed; the next most commonly 
reported rationale was an interest in creating an international environment in their research 
group (60 percent). 
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3.2 Applications and Awards 


According to the program solicitation, IRFP fellowships fund research projects with international 
collaborations that are expected to benefit the applicant, the research discipline, and scientists and 
engineers in each country. NSF's program data reveal that IRFP is a fairly selective program, having 
accepted approximately one-third of applications. As shown in Exhibit 3.1, the program has steadily 
become more competitive, despite year-to-year fluctuations. From 1992 to 2009, NSF program data 
indicate that the program has received a total of 1,660 applications, with annual applications 
ranging from 35 (in 1992) to well over 100 (after 2000). The acceptance rate ranges from 63 percent 
(in 1992) to 21 percent (2004 and 2005), and across the program's history, averages 35 percent. 


Exhibit 3.1: Number of IRFP Applications and Percent Awarded, 1992-2009 


Year 

N Received 

Percent Awarded 

1992 

35 

62.9 

1993 

44 

52.3 

1994 

40 

60.0 

1995 

115 

27.8 

1996 

76 

35.5 

1997 

84 

31.0 

1998 

54 

38.9 

1999 

73 

42.5 

2000 

70 

37.1 

2001 

119 

37.8 

2002 

122 

33.6 

2003 

122 

28.7 

2004 

167 

21.0 

2005 

156 

20.5 

2006 

177 

20.9 

2007 

159 

24.5 

2008 

129 

24.8 

2009 

121 

44.6 

NOTES: Each application received is counted, such that some individuals who applied for IRFP more than once are 
included in multiple years; an individual may receive only one IRFP award. 

SOURCE: NSF Extant Data, SDR 2008. 

3.3 Characteristics and Background of Applicants 


3.3.1 IRFP Applicants: Selected Characteristics 

Individuals who apply for IRFP represent a diverse and qualified pool of scientists. In this section, we 
describe the background characteristics of these applicants. 


Demographics 

Exhibit 3.2 and 3.3 present the gender and race/ethnicity of applicants, both overall and among 
those who received fellowships and those who were unfunded. The majority of applicants were 
male (61 percent), and a majority were either White non-Flispanic (86 percent) or Asian (5 percent). 
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Seven percent were members of racial or ethnic groups traditionally underrepresented in STEM: 2 
percent Black/ African-American, 5 percent White-Hispanic, and less than 1 percent each were 
American Indian or Alaska native or Native Hawaiian/other Pacific Islander. At the time of the 
survey, 2 percent of individuals who had applied to IRFP reported a disability (0.4 percent of IRFP 
awarded applicants; 2 percent of unfunded applicants; data missing for 14 applicants, 7 of whom 
were former awardees). 47 

Exhibit 3.2: Gender of IRFP Applicants 

0 25 50 75 100 


61.2 

62.0 

60.7 

■ All Applicants (N=1039) 

■ Awarded (N=457) 

■ Unfunded (N=582) 


SOURCE: IRFP Applicant Survey-Item Gl, NSF Extant Data. 



Respondents who indicated that they were unable to see (with glasses or contact lenses if usually worn), 
hear (with hearing aid if usually worn), walk without human or mechanical assistance, or lift 10 pounds, or 
who reported moderate or severe difficulty with these abilities were classified as disabled. No respondent 
indicate that they were unable to see, hear, walk or lift. 
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Exhibit 3.3: Race/Ethnicity of IRFP Applicants 


White, 

Non-Hispanic 

Asian 

Hispanic 

Multi 
Black or 

African American 

American Indian or 
Alaska Native 

Native Hawaiian or 
other Pacific Islander 


NOTES: Individuals who identified as Hispanic are grouped together regardless of their racial identification. All racial 
categories exclude individuals who identified as Hispanic. Missing Data: 18 Awarded, 32 Unfunded Applicants. 

SOURCE: IRFP Applicant Survey-Item G2, NSF Extant Data. 

Field of Research 

The disciplinary backgrounds of applicants spanned the range of STEM disciplines represented by 
NSF directorates (Exhibit 3.4). The largest proportion of applicants had training in life science 
disciplines (including biological, agricultural and environmental life sciences) (47 percent), followed 
by physical sciences (26 percent). Fewer applicants were conducting research in engineering (15 
percent), and the remaining 12 percent of applicants were conducting research in computer 
sciences and mathematics (6 percent), and social sciences (6 percent). 48 


0 25 50 75 100 



This study used the same classification for field of research employed by the Scientists and Engineers 
Statistical Data System (SESTAT). 
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Exhibit 3.4: Field of Research at Time of IRFP Application 


Biological/Life sciences 


Physical sciences 


Engineering 


Computer/ 
Information sciences. 
Mathematics/ Statistics 


Social sciences 



SOURCE: IRFP Applicant Survey-Item AS, NSF Extant Data. 


Applicants’ Qualifications and Credentials 

The IRFP program targets early career recent doctoral recipients. Although applicants were not 
required to have received their PhD at the time of application, completion of a PhD was required 
prior to program participation; the maximum post-PhD time allowed varied by solicitation and 
ranged from two to six years. At the time of their application, 58 percent of IRFP applicants had 
earned their PhD and 6 percent were employed in a tenure-track position. Among applicants who 
subsequently received an IRFP award (i.e., IRFP fellows), a smaller proportion (54 percent) had 
completed their PhD at the time of application compared to applicants who were not funded (60 
percent). 


On average, awarded and unfunded IRFP applicants shared similar academic credentials at the time 
of application. Forty-five percent had received a nationally competitive fellowship 49 to support their 
graduate studies (Exhibit 3.5). 


49 Competitive funding included graduate funding awarded on a competitive basis directly to an individual 
for use at any graduate institution. Funding from a grant that was awarded to an institution or faculty 
member was not included. 
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Exhibit 3.5: Credentials of IRFP Applicants at the Time of IRFP Application 


100 

80 

60 

40 

20 

0 


■ All Applicants 

■ Awarded 


50.4 ■ Unfunded 



Had completed doctoral Had received a nationally Was employed in a tenure 
degree competitive graduate track position 

fellowship 


NOTES: IRFP applicants who had not yet officially received their PhD could apply to the program if they expected to 
receive their PhD by the start of their awarded fellowship. Completed doctorate: N=1035 (456 Awarded, 579 Unfunded 
Applicants), Missing=4 (1 Awarded, 3 Unfunded Applicants); Graduate fellowship: N=1026 (454 Awarded, 572 Unfunded 
Applicants), Missing=13 (3 Awarded, 10 Unfunded Applicants); Tenure Track: N=1037 (457 Awarded, 580 Unfunded 
Applicants), Missing=2 Unfunded Applicants. 

SOURCE: IRFP Applicant Survey-Items C2a; C8; Cl. 


On average, applicants reported having had an average of 9.5 publications at the time of their 
application (Exhibit 3.6). Relatively few applicants reported that they held patents (a mean of 0.1 
patents across all applicants). Few of these publications (16 percent) and patents (less than 1 
percent) were with foreign collaborators, although slightly more of the patents of applicants 
awarded an IRFP fellowship than unfunded applicants were with a foreign collaborator (0.8 percent 
versus 0.1 percent). Although the percentage of publications with foreign collaborators was low, 
survey responses provided evidence that applicants had some international experiences and some 
had already started to explore international research networking prior to application. 
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Exhibit 3.6: Average Number of Publications and Patents, and Percent Co-authored With Foreign 
Collaborator at Time of IRFP Application 


Publications and Patents 

All 

Applicants 

Awarded 

Unfunded 

Mean number at time of application 

N 

N 

N 

Publications 

9.5 

9.7 

9.3 

Patents 

0.1 

0.2 

0.0 

Of the publication and patents, percent produced with a 
foreign collaborator 

% 

% 

% 

Mean percent of publications 

16.0 

16.5 

15.6 

Mean percent of patents 

0.4 

0.8 

0.1 


NOTES: Publications included peer-reviewed journal articles, peer-reviewed conference publications and book 
chapters; patents included registered or pending. Missing Data: 11 to 18 Awarded Applicants; 28 to 35 
Unfunded Applicants. 

SOURCE: IRFP Applicant Survey-Item C7. 

Applicants’ Prior International Experience 

For some applicants, the IRFP extends previous international experiences, including academic and 
professional experiences (Exhibit 3.7). For example, 47 percent of IRFP applicants had participated in 
an undergraduate or graduate study abroad program or had pursued a graduate degree outside the 
U.S. Further, 44 percent had lived outside the U.S. for six months or more. 

There was also evidence that applicants were beginning to explore or build professional 
relationships or seek networking opportunities with scientists in foreign locations. At the time they 
applied, 60 percent of IRFP applicants had attended or presented their own research at a 
professional conference outside the U.S. Further, more than half (53 percent) had collaborated on 
research with someone from another country, and nearly one-third (33 percent) had published 
research with someone based in another country. These types of previous international experiences 
may have sparked some applicants' interest in a postdoctoral fellowship abroad. 
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Exhibit 3.7: International Experiences Prior to IRFP Application 



Pre-Application International Experience(s) 

All Applicants 

Awarded 

Unfunded 

Undergraduate or graduate education abroad 3 

46.5% 

47.4% 

45.9% 

Lived outside the U.S. for > 6 months 11 

43.8 

45.8 

42.6 

Attended elementary or secondary school in another 
country 11 

14.3 

13.9 

14.5 

Attended or presented scholarly work at a research 
conference in another country 11 

60.3 

61.1 

59.8 

Collaborated on research with someone based in 
another country 11 

53.1 

55.6 

51.6 

Published research with someone based in another 
country 11 

32.8 

34.1 

32.0 

Participated on research team with a scientist visiting 
from foreign institution 11 

36.0 

35.2 

36.4 

Worked with former IRFP fellow b 

3.1 

3.4 

3.0 


3 Includes undergraduate study abroad (C3a), graduate study abroad (C3b) and pursued a graduate degree abroad (C2b). 
Missing Data: 0 to 3 Awarded Applicants; 4 to 7 Unfunded Applicants. 
b Missing Data: 2 Awarded, 2 Unfunded Applicants 
SOURCE: IRFP Applicant Survey-Items C2b, C3a-3b, C4. 

Although a sizeable percentage of IRFP applicants had some prior international experiences in 
general, fewer IRFP applicants had a preexisting relationship with the particular host scientist with 
whom they proposed to collaborate (Exhibit 3.8). Overall, 28 percent of applicants were already 
collaborating with their proposed host scientist at the time of application (and another 6 percent 
were collaborating with someone at the same institution as their proposed host). Flowever, less 
awarded than unfunded applicants reported such a preexisting collaboration with their host (21 
percent of awarded versus 32 percent of unfunded applicants). Similarly, while 14 percent of 
applicants overall were already working at their proposed host's institution, this was less often the 
case for those awarded IRFP fellowships (9 percent) than applicants who were not funded (17 
percent). These data appear to reflect a program emphasis on sparking new international 
collaborations rather than reinforcing existing collaborations. 
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Exhibit 3.8: IRFP Applicants' Pre-application Relationship with Host Scientist or Host's Institution 


■ All Applicants 

■ Awarded 

■ Unfunded 


16.5 


9.0 


Already collaborating with host Already collaborating with Already working at host 

someone else at host institution institution 


NOTES: Already collaborating with host, or with someone else at host institution: N=l,033 (455 Awarded, 578 Unfunded 
Applicants), Missing=6(2 Awarded, 4 Unfunded Applicants); Already working at host institution: N=l,034 (456 Awarded, 
578 Unfunded Applicants), Missing=5 (1 Awarded, 4 Unfunded Applicants). 

SOURCE: IRFP Applicant Survey-Items C5a-5b. 

3.3.2 Hosts’ Prior Experience with the U.S. 

The scientists and engineers who hosted IRFP fellows were more likely to be familiar with science 
and engineering in the United States. Although 66 percent of IRFP host scientists had never hosted a 
postdoctoral fellow from the U.S. prior to hosting an IRFP-sponsored fellow (no exhibit), the 
majority of IRFP host scientists and engineers had studied, taught, or conducted research in the U.S. 
before hosting an IRFP fellow (Exhibit 3.9). Just over half had been a visiting scientist in the U.S. Less 
than 3 percent of IRFP hosts had never visited the U.S. in a professional capacity prior to hosting an 
IRFP fellow. 


Exhibit 3.9: Host Scientists' Experiences in the U.S. Prior to Hosting an IRFP Fellow 


Hosts' Pre-IRFP Experiences in the U.S. 

Percent of IRFP 
Hosts (N=328) 

Attended a conference, workshop or meeting in the U.S. 

75.0 

Was a visiting scientist in the U.S. 

50.6 

Was a postdoctoral fellow in the U.S. 

35.1 

Was a graduate student in the United States 

24.4 

Was a faculty member in the U.S. 

18.3 

Was an undergraduate student in the U.S. 

7.6 

Other visit to the U.S. for educational, research or professional purposes 

7.3 

Had not visited the U.S. for professional purposes before hosting an IRFP fellow 2.4 

NOTE: Responses do not sum to 100 percent because multiple choices were permitted. 
SOURCE: IRFP Host Survey-Item B5. 

Missing Data: 0 Hosts. 



13.7 


6.0 6.1 


5.3 
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Preferences, Support, and Motivation 


Applicants were asked to describe the support they received from their U.S. advisors, as well as to 
report the reasons for having applied to IRFP and having selected a particular host location. Foreign 
hosts were also asked to describe their motivations for participating in the IRFP program. Findings 
from these responses provide valuable insight into the motivations of individuals, although it is 
important to note that the data also reflect respondents' retrospective memories about attitudes 
and beliefs that may reflect distortions of time, particularly when the outcomes of events are 
known. 50 

3.4.1 Faculty Advisors’ Support for IRFP Applicants 

According to applicants' survey responses, graduate advisors were supportive of their decisions to 
apply for IRFP, and many were active participants in the application process (Exhibit 3.10). The 
majority of applicants' graduate advisors supported (51 percent) or actively encouraged (21 percent) 
their students' decision to apply for an IRFP postdoctoral fellowship. Further, 49 percent of 
applicants received feedback from their advisor on their IRFP proposal, and 24 percent received a 
recommendation from their advisor to a colleague at the host institution. 


Exhibit 3.10: Advisor's Support for Decision to Apply for IRFP 



All Applicants 

Awarded 

Unfunded 


(N=l,037) 

(N=457) 

(N=576) 

Support from faculty advisor 3 


Advisor supported decision to apply 

51.2% 

51.6% 

51.0% 

Advisor encouraged me to apply to the IRFP program 

21.0 

22.0 

20.4 

Don't know how my advisor viewed my decision to apply 

13.7 

11.6 

15.0 

Advisor was indifferent to my decision to apply 

6.1 

7.6 

5.1 

1 did not have an advisor when 1 most recently applied 

6.0 

6.2 

5.8 

My advisor opposed my decision to apply 

2.0 

1.0 

2.7 

Mentoring or guidance from faculty advisor” 

Provided feedback on my project proposal 

48.6% 

54.2% 

45.1% 

None 

38.2 

32.3 

41.7 

Recommended me to a colleague at the host institution 

24.2 

24.3 

24.2 

Suggested a host institution 

15.3 

14.5 

15.8 

Other 

5.2 

6.0 

4.8 

Discussed cultural, language aspects of the host country with 
me 

4.9 

6.6 

3.9 

NOTE: Results do not sum to 100 percent because multiple choices were permitted. 
a N=1033 (457 Awarded, 576 Unfunded Applicants), Missing=6 Unfunded Applicants. 


b N=1032 (456 Awarded, 576 Unfunded Applicants), Missing=7 (1 Awarded, 6 Unfunded Applicants). 

SOURCE: IRFP Applicant Survey-Items B4 and B5. 

Applicants’ Proposed Host Countries 

In addition to drawing candidates with a strong interest in and potential for successful international 
collaboration, applicants' preferences for host sites reflected a broad geographic distribution, 


Hawkins, S. A., & Hastie, R. 1990. Hindsight: Biased judgments of past events after the outcomes are 
known. Psychological Bulletin, 107(3), 311-327; Stahlberg, D., & Maass, A. 1998. Hindsight bias: Impaired 
memory or biased reconstruction? European Review of Social Psychology, 8, 105-132. 
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although the applicant pool showed preference for regions with more developed research 
infrastructures (Exhibit 3.11). Most commonly, host institutions proposed by applicants were in 
Europe (60 percent), followed North America (10 percent), and the South Pacific (9 percent), which 
included Australia, New Zealand, Papua New Guinea, Fiji Islands, Palau, and the Solomon Islands. 

Applicants who proposed to work with a host in South or Central America had the highest success 
rate (47 percent of applicants to this region were funded), followed by Europe (41 percent); a 
smaller proportions of applications to other North American countries (i.e., Canada or Mexico: 31 
percent) and Africa or the Middle East (31 percent) were funded. 


Exhibit 3.11: Geographic Region of Proposed Host Site Country and Success Rate of Applications 



% of Applicants to 
Region 

% Successful within 
Region 

Europe 

60.2 

40.8 

North America 

10.4 

30.9 

South Pacific 

9.4 

38.3 

South and Central America 

9.4 

47.1 

Africa/Middle East 

7.2 

31.2 

East Asia 

5.0 

36.2 


SOURCE: IRFP Applicant Survey-Item A3; NSF Extant data 

3.4.2 Applicants’ Motivations 

IRFP applicants generally cited multiple reasons for 
applying to IRFP (Exhibit 3.12), and some reasons were 
commonly mentioned. More than four-fifths cited a 
desire to work with a specific person or at a specific 
institution and to enhance their research knowledge 
and skills (87 and 82 percent, respectively). About 
three-fifths indicated that they wanted to collaborate 
with foreign scientists generally (59 percent), and a 
majority believed that an IRFP award would enhance 
their resumes (57 percent). Just under half also 
expressed the appeal of traveling outside the U.S. (44 
percent), the desire to learn about another country (42 
percent), and understanding what their research field 
was like outside the U.S. (40 percent). Less than one-third noted that they were motivated by a 
desire to access a particular resource (29 percent). A greater proportion of awarded than unfunded 
applicants cited a desire to learn about how research in their field was conducted outside of the U.S. 
(49 versus 35 percent, respectively) and to gain access to resources that were unavailable in the U.S. 
(31 versus 27 percent). 

Applicants' reported motivations for applying may not have been particularly relevant to their 
award outcome, although it is possible that some of their motivations were, at least implicitly, 
conveyed in their proposals. Slightly more awarded than unfunded applicants thought that an IRFP 
award would enhance their resumes (61 versus 54 percent). Awardees were also more likely than 


[The IRFP Fellowship] allowed me 
to [do] research that was possible 
only by working in collaboration 
with the host institution and 
conducting fieldwork in the host 
country. I chose this country 
specifically because regional 
biogeography allowed me to follow 
the path ofinguiry that I had 
hoped to examine. (IRFP Applicant) 
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unfunded applicants to report cultural motivations for applying (e.g., desire to learn about the 
culture, history, geography or language of another country). 


Exhibit 3.12: Applicants Reasons for Applying to IRFP 


0 20 40 60 80 100 


To conduct research with a specific person or at a 
specific institution 


To enhance my skills or knowledge as a researcher 
To collaborate with a foreign scientist 
To enhance my resume as a future job candidate 
To travel outside the US 


To learn about the culture, history, and/or 
geography of another country 

To understand what research in my field was like 
outside the US 


To access resources 


To learn another language 


For family reasons 


86.7 
86.4 

86.8 



All Applicants (N=l,037) 
Awarded (n=457) 
Unfunded (n=580) 


NOTES: Results do not sum to 100 percent because multiple choices were permitted. Missing Data: 2 Unfunded 
Applicants. 

SOURCE: IRFP Applicant Survey-Item Bl. 
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Applicants also cited multiple motives for proposing a 
particular host country (Exhibit 3.12); a large majority 
reported that they wanted to work with a particular host 
located in that country (87 percent), and over half noted 
that the host's institution had equipment or resources 
that would benefit their research (52 percent). Less than 
50 percent of applicants cited any other single reason 
for selecting a particular host site. 


the chance to work with ... 
scientists [in my host country] 
during the 1990s, an exciting period 
of change when the involvement of 
[scientists in my host country] in 
field ecology and conservation was 
just at its initial stages. (IRFP 
Applicant) 


Awarded fellows may have benefited from closer 
alignment between their own research and their 
respective proposed hosts' interests and resources: a higher percentage of IRFP awardees than 
unfunded applicants indicated that the host's research was relevant to their own (90 versus 85 percent), 
and that the host's institution had resources beneficial to their research (56 versus 49 percent). In 
addition, more IRFP awardees than unfunded applicants explained that they selected their proposed site 
because they had professional ties with someone in that country, and more awarded fellows cited a 
desire to visit particular places in the host country. 
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Exhibit 3.13: Applicants Reasons for Selecting a Specific Host Country 


0 


50 


100 


The host was conducting research relevant to my 
own interests 

The host institution had ... resources helpful for my 
... research 

I have professional ties with someone who is from 
this country 

A faculty advisor or mentor recommended this 
site/host 

I was familiar with the primary language(s) spoken 
there 

The host researcher had visited my department or 
university 

There were places in this country that I wanted to 
visit or see 

I had studied this country's culture, history, politics, 
geography, etc. 


Other 



NOTES: Results do not sum to 100 percent because multiple choices were permitted. Missing Data: 2 Unfunded Applicants. 
SOURCE: IRFP Applicant Survey-Item B2. 


3.4.3 Hosts: Motivations and Concerns 


Foreign scientists with whom IRFP fellows worked cited a variety of motives for hosting an IRFP fellow, 
most frequently having interest in the research project that the fellow had proposed (83 percent of host 
researchers) (Exhibit 3.14). Another common motive, cited by 60 percent of hosts, was a desire to create 
an international environment in their research group; 44 percent cited a general interest in establishing 
a collaboration with someone in the U.S. The host's prior knowledge of the fellow or the fellow's advisor 
also played a role for more than 40 percent of hosts. 
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Exhibit 3.14: Hosts' Motivations for Hosting the IRFP Fellow 



Percent of Hosts 


(N=328) 

1 was interested in the project proposed by the fellow 

82.9 

To create an international environment in my research group 

59.5 

To attract students/postdocs to my research 

48.8 

1 personally knew, knew of, or previously collaborated with the fellow 

47.0 

1 was interested in establishing or maintaining collaboration with a U.S. 
researcher 

43.9 

1 personally knew, knew of, or previously collaborated with fellow's doctoral 
advisor 

43.3 

To learn new methodologies, approaches, or tools from the fellow 

35.4 

1 had a positive experience with other U.S. postdoctoral or visiting researchers 
(not IRFP-funded) 

30.5 

1 personally knew, knew of, or previously collaborated with researchers at fellow's 
institution 

20.7 

1 had a positive experience with the IRFP program in the past 

4.6 


NOTES:. Responses do not sum to 100 percent because multiple choices were permitted. Missing Data: 0 Hosts 
SOURCE: IRFP Host Survey-Item B2. 


The majority of hosts reported no specific concerns about collaborating with an IRFP fellow (77 percent). 
The percentage of hosts who indicated one or more concerns was less than 10 percent (Exhibit 3.15). 


Exhibit 3.15: Hosts' Concerns about Hosting the IRFP Fellow 



Percent of Hosts 
(N=328) 

1 was concerned about the integrating this postdoctoral fellow into my research 
group 

6.4 

1 was concerned about the fellow's level of commitment to a collaboration with me 

5.2 

Other concerns not listed 

3.7 

The fellow's proposed project was especially risky 

3.7 

1 was concerned that 1 might not (or my research group might not) benefit from 
hosting this fellow 

3.0 

In my field, individuals trained at U.S. graduate institutions sometimes have gaps in 
their knowledge, skills, or abilities 

2.4 

1 was concerned about the risks of international collaboration in general 

1.8 

1 had a negative experience with the IRFP program in the past 

0.3 

No concerns indicated 

77.2 


Missing Data: 0 Hosts. 

SOURCE: IRFP Host Survey-Item B3. 
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4 Fellowship Experience 


This chapter describes the IFRP fellowship experience 
from the perspectives of both the IRFP fellows and the 
research scientists who hosted them. Findings include 
reflections on the fellows' engagement in research and 
cultural activities, fellows' and hosts' perception of 
aspects of their research collaboration, fellows' and 
hosts' satisfaction with various aspects of their IRFP 
experience, and perceived challenges of the fellowship. 

Specifically, this chapter addresses the following 
question: 

• What are the program experiences of program 
participants? 


[IRFP] is an experience which is 
culturally enriching, and 
scientifically productive. Research 
at the forefront of science requires 
combining the best researchers 
and facilities on the world. It is 
often necessary to combine 
equipment and expertise by people 
on different continents, if one 
really wants to get the best 
possible results. (IRFP Host) 


Findings in this chapter are drawn from 457 IRFP fellows 
who completed the applicant survey and 328 foreign researchers who responded to the IRFP host 
survey. As explained in the methodology section, percentages for fellows are weighted to adjust for 
nonresponse, while reported n's are not weighted. Differences between fellows by geographical 
location of their fellowship are described where statistically significant. 


4.1 Key Findings 


• Former fellows and hosts offer strong endorsement for the IRFP program. All former IRFP 
fellows would recommend the program to a colleague, and the large majority of hosts (84 
percent) would recommend, or have already recommended, to others that they host an 
IRFP postdoctoral fellow from the U.S. 

• Hosts and fellows agreed they worked collaboratively on developing project ideas and 
hypotheses, interpreting results, and planning follow-up work. 

• Hosts and fellows agreed that fellows worked mostly independently collecting data, carrying 
out simulation, and keeping records or tracking supplies and resource. 

• Over 90 percent of fellows were somewhat or very satisfied with the quality of research 
they were able to conduct as part of IRFP (92 percent) and the timing of the fellowship with 
respect to their career goals (97 percent). 

• Other professional activities featured prominently in the experiences of many fellows. 
Specifically, IRFP provided fellows with the opportunity to network with colleagues from 
institutions outside their host institution (90 percent), attend lectures in their field (89 
percent), visit other institutions (89 percent), attend conferences (84 percent), and give 
talks at their institution (71 percent). 
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• Fellows also experienced a variety of cultural activities including sightseeing (96 percent) 
outdoor activities that explore geography (95 percent), museums (89 percent), and festivals 
or holidays (85 percent). 

• Forty percent of fellows reported no noteworthy difficulties during their fellowship. The 
most common challenges were related to logistics (32 percent) and language difficulties (20 
percent), followed by inadequate access to space or resources (16 percent), and not enough 
guidance from the host researcher or research group (15 percent). 

• Over 80 percent of fellows were satisfied with many aspects of their host, including the 
match between research interests, the host's expertise and contributions to the research, 
their inclusion in collaborations, and frequency of meetings. 

• Most hosts (75 percent) reported no difficulty in hosting an IRFP fellow. Specific challenges 
were identified by less than 10 percent of hosts, including noting that the fellow did not 
devote enough time to the research collaboration (10 percent), and the fellow worked too 
independently or did not work well as a collaborator (8 percent). 

• Hosts noted that IRFP fellows compared favorably to other postdoctoral fellows with whom 
hosts had worked; 56 percent of hosts were much or somewhat more satisfied with IRFP 
fellows compared to other postdoctoral fellows, and another 28 percent were equally 
satisfied. 

• Only one-third of hosts reported any challenges experienced in hosting an IRFP fellow. 

• Hosts generally strongly agreed or agreed (72 and 24 percent, respectively) that their IRFP 
fellow had sufficient knowledge and expertise, and that the fellow integrated well with their 
research group (66 and 26 percent, respectively). 


4.2 Program Recommendation 


Former fellows and hosts offered strong endorsement for the IRFP program. One hundred percent 
of former IRFP fellows would recommend the program to a colleague; 97 percent would 
recommend their host country; and 79 percent would recommend their host scientist to a 
colleague. Among hosts to IRFP fellows, 84 percent would recommend, or have already 
recommended, to others that they host an IRFP postdoctoral fellow from the U.S.; another 14 
percent indicated that they might recommend hosting an IRFP fellow depending on the 
qualifications of the individual candidate. 


4.3 Geographic Locations 


Between 1992 and 2009, the majority of IRFP fellows spent their postdoctoral appointments in 
Europe (63 percent, Exhibit 4.1). Fewer conducted research in South or Central America (9 percent), 
the South Pacific (e.g., Australia, New Zealand, Palau, Papua New Guinea, or the Solomon Islands; 9 
percent), Canada or Mexico (8 percent), Africa or the Middle East (6 percent), and East Asia (China, 
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India, Indonesia, Malaysia, Mongolia, Pakistan, Philippines, Singapore, South Korea, Sri Lanka or 
Taiwan; 5 percent). 51 

Exhibit 4.1: Regional Location of IRFP Fellows' Host Institutions, 1992-2009 

Percent 



SOURCE: IRFP Applicant Survey-Item A3. 

4.3.1 Language Preparation 

In preparation for their internationally based postdocs, 42 percent of fellows completed some form 
of language study (Exhibit 4.2), including self-guided study (28 percent), formal courses (18 percent), 
and tutoring (7 percent). Many fellows (44 percent, no exhibit) indicated that they were already 
familiar with the language spoken in the host country. 

Although 21 percent of fellows indicated that they experienced communication or language 
difficulties during their fellowship (see "Barriers Encountered" below), only a fraction of host 
scientists reported that language differences posed a challenge: 7 percent agreed or strongly agreed 
that language differences were a barrier to their fellow's ability to interact with their research 
group, and just 1 percent reported that their IRFP fellow's lack of familiarity with their language 
made collaboration more difficult than they had expected. 


Exhibit 4.2: Language Study in Preparation for IRFP Fellowship 



Percent of IRFP 
Fellows (N=450) 

No language study 

58.2 

Any language study 

42.3 

Yes, some self-guided language study 

28.4 

Yes, formal language training course 

17.6 

Yes, study with tutor or conversation partner 

6.8 


NOTES: Results do not sum to 100 percent because multiple choices were permitted. Missing Data: 7 Fellows. 
SOURCE: IRFP Applicant Survey-Item El. 


The locations in Exhibit 3.11 were the host location applicants proposed, while the locations in Exhibit 4.1 
represent the actual host locations of awardees. 
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4.4 Fellows’ Experience 


4.4.1 Fellow-Host Collaboration 

As described in Chapter 3, for most fellows the primary reason to pursue an IRFP fellowship was to 
conduct research with a particular host scientist or at a particular institution. Of particular interest, 
then, is the fellows' experience collaborating on various aspects of their research with their host. 
Nearly three-fourths (73 percent) of fellows indicated that the host researcher provided direct 
supervision and 22 percent indicated that they were not supervised by anyone (Exhibit 4.3). 


Exhibit 4.3: Individual(s) Who Provided Direct Supervision to IRFP Fellows During Fellowship 



Percent of Fellows 
(N=448) 

The host researcher him/herself 

73.1 

No one: 1 was not supervised by anyone 

21.8 

Another staff scientist 

13.3 

A junior faculty member or postdoctoral fellow 

7.7 

A laboratory technician or other employee/worker 

4.9 

Another graduate student(s) 

3.3 


NOTES: Results do not sum to 100 percent because multiple choices were permitted. Missing Data: 9 Fellows. 
SOURCE: IRFP Applicant Survey-Item E6. 


Variation in the host supervisor's role may well reflect the fellow's level of expertise or desire for 
independence, the nature of the research, and the size of the research team involved, among other 
factors. As one fellow commented. 

My host functioned as an advisor. He was present whenever I needed advice. Otherwise, I 
functioned rather autonomously. I consulted with my host as well as other individuals ... mostly 
to avoid cultural conflicts as well as to benefit from their respective knowledge. (IRFP fellow) 

Comments provided by fellows illustrate the value of their hosts in their IRFP experiences: 

My weekly meetings with my host supervisor were consistently inspiring and extremely 
productive in pursuing my research. They will have a lasting impact on how I do my research. 
(IRFP fellow) 

My best scientific work has been done with [my former host] and most of what I know about 
how to be a good scientist I learned from him. He continues to be a valued mentor to me. (IRFP 
fellow) 

The IRFP fellowship gave me the opportunity to work with one of the most well-respected 
researchers in my field on a one-to-one basis. I have become a better scientist and gained a 
valuable mentor and friend through this experience. (IRFP fellow) 

Additional descriptive understanding of fellows' interactions with their hosts emerges from an 
examination of fellows' and hosts' perceptions of their contributions across a range of research 
activities and tasks (Exhibit 4.4). 

The majority of fellows reported that they worked independently of their host for many activities. 
Interestingly, the majority of hosts were in disagreement with the majority of fellows. The hosts 
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reported that they and their fellow contributed about equally to the same activities. This apparent 
discrepancy between fellows' and hosts' perceptions likely results, in part, from two common 
cognitive biases affecting autobiographical memory. In a range of group settings, individuals inflate 
their own contribution to a joint task in achieving a goal relative to other members of the group. 52 In 
addition, individuals' perceptions of their behaviors tend to be biased in such a way that they see 
themselves acting in accordance with an internal schematic encompassing their motives and goals. 53 
Thus, it is likely that fellows perceived their contributions as favoring their developmental goal of 
establishing themselves as independent researchers, whereas hosts were likely to perceive their 
contributions as favoring their motivation to be an effective mentor, nurturing the fellow's 
professional development through extensive collaboration. 

Nevertheless, looking across fellows' and hosts' perceived contributions across several activities, a 
picture emerges of which activities were characterized as most and least collaborative by both 
fellows and hosts. For example, fellows and hosts generally agreed that certain research activities 
were more collaborative than other activities, including developing project ideas and hypotheses, 
interpreting results, and planning follow-up work (Exhibit 4.4, top and bottom panels). The majority 
of both fellows and hosts credited fellows with working mostly independently to collect data or 
carry out simulations (66 percent of fellows and 58 percent of hosts) and to keep records or track 
supplies and resources (64 percent of fellows and 57 percent of hosts). In addition, for all but one 
activity (developing instrumentation, software, equipment or other data collection processes) less 
than 10 percent of fellows or hosts reported that the host was primarily involved without input from 
the fellow. 

There was less agreement between fellows and hosts about the degree of collaboration in 
researching relevant literature and analyzing data. Whereas 47 percent of hosts believed that they 
and the fellow were equally involved in researching extant literature related to their project, only 27 
percent of fellows reported equal collaboration in this activity. Similarly, 43 percent of hosts 
reported that they and their fellow were equal contributors to data analyses, but less than 26 
percent of fellows perceived equal engagement in this activity. Again, differences between fellows 
and hosts may well reflect individuals' (whether the fellow's or the host's) tendency to overestimate 
their own role in a collaborative effort. 


Kruger, J., & Savitsky, K. 2009. On the genesis of inflated (and deflated) judgments of responsibility: 
Egocentrism revisited. Organizational Behavior and Human Decision Processes, 108 (1), 143-152; Ross, M. 
& Sicoly. F. 1979. Egocentric biases in availability and attribution. Journal of Personality and Social 
Psychology, 37, 322-336. 

Woike, B. 2008. A functional framework for the influence of implicit and explicit motives on 
autobiographical memory. Personality and Social Psychology Review, 12(2), 99-117Conway, M.A. & 
Pleydell-Pearce, C.W. 2000. The construction of autobiographical memory in the self-memory system. 
Psychological Review, 107(2), 261-288. 
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Exhibit 4.4: Fellows' and Hosts' Perception of Who was Primarily Involved in Activities Related to 
the IRFP Research Project 
Fellows' Perception 


0 20 40 60 80 100 

Planning/developing follow-up work 
Interpreting results 
Developing project ideas, hypotheses, framework, vision 

Dissemination of results (publications, presentations) 

Developing instrumentation, software, equipment, or data 
collection processes 

Researching the relevant literature/research 
Analyzing data, observations 
Collecting data, carrying out simulations 
Keeping records, tracking supplies, resources 

J I I I I I 

■ Fellow & Host Equally ■ Mostly the Fellow ■ Mostly the Host ■ N/A 



Hosts' Perception 


0 20 40 60 80 100 


Planning/developing follow-up work 

Interpreting results 

Developing project ideas, hypotheses, framework, 
vision 

Dissemination of results (publications, 
presentations) 

Developing instrumentation, software, equipment, 
or data collection processes 

Researching the relevant literature/research 
Analyzing data, observations 
Collecting data, carrying out simulations 
Keeping records, tracking supplies, resources 

■ Fellow & Host Equally ■ Mostly the Fellow 

■ Mostly the Host ■ N/A 



3.5 
0.3 

1.5 


0.9 

5.4 


IRFP Fellows: N=457, Missing: 9 to 14 Fellows. IRFP Hosts: N=328, Missing: 3 to 14 Hosts. 
SOURCE: IRFP applicant survey-item E4; IRFP host survey-item Cl. 
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4.4.2 Professional and Cultural Activities 


In addition to pursuing their primary research 
with their host, fellows engaged in many 
other professional activities (Exhibit 4.5), 
including networking with colleagues from 
institutions other than their host's (90 
percent), attending lectures, seminars, and 
colloquia in their field (89 percent), attending 
professional conferences in their host country 
(84 percent), giving an oral presentation to 
researchers at their host institution (71 
percent), and visiting businesses or industrial sites 


One of my most memorable activities was a trip [to 
another city in Europe] to attend a workshop.... It 
was very soon after my arrival to my host laboratory. 
At the workshop I was able to meet and learn from 
experts in my field from all over Europe [and] I was 
honored to give a talk describing my doctoral 
research. This is something I never would have done 
if I had not been working in Europe. (IRFP fellow) 


relevant to their research interests (23 percent). 


Exhibit 4.5: Professional Activities of IRFP Fellows During Fellowship 


1 Professional Activities 

Percent of Fellows (N=450) 1 

Networking with colleagues from institutions other than host 

90.3 

Lectures, colloquia, seminars in my field 

89.4 

Visit(s) to educational or research institutions other than host 

88.5 

Attended professional conferences in host country 

84.0 

Gave talk or presentation to researchers in host site 

71.1 

Visit(s) to businesses/industrial laboratories 

22.5 

Other 

3.6 

NOTES: Results do not sum to 100 percent because multiple choices were permitted. 
Applicants. 

SOURCE: IRFP applicant survey-item E3. 

Missing Data: 7 Awarded 


Living outside the United States was 
both technically and cultural [sic] 
rewarding. Although I went to become a 
better scientist, being introduced to 
another language, a different way of life 
and new traditions was far more 
enriching. (IRFP fellow) 


Activities abroad also included cultural activities, which 
many fellows reported to be rewarding (Exhibit 4.6). 
Over one-third of fellows (34 percent) attended 
language courses or engaged in some other type of 
language study during their fellowship. The most 
frequently cited leisure activities included sightseeing 
(96 percent), outdoor activities (95 percent), visiting 
museums (89 percent) and attending festivals, holiday 
events or religious ceremonies (85 percent). 


Exhibit 4.6 Cultural Activities during Fellowship 


j Cultural and Leisure Activities 

Percent & 

Sightseeing 

96.3 

Outdoor activities to explore the landscape/geography 

95.4 

Museums 

89.1 

Festivals/Holidays/Religious ceremonies 

85.1 

Non-scientific lectures or presentations 

57.3 

Sporting events 

48.3 

Language courses or language study 3 

34.3 

Other 

7.6 

No participation in cultural activities 

0.6 

3 This question asked about language study during fellowship, while Exhibit 4.2 
for fellowship. 

SOURCE: IRFP applicant survey-item E3. 

included language study in preparation 
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4.4.3 Barriers Encountered 

Forty percent of IRFP fellows reported that they had encountered no difficulties during their 
fellowship (Exhibit 4.7). Responses from those who cited barriers, however, noted logistical 
challenges (32 percent), such as transportation or navigating through foreign bureaucracies, and 
communication or language difficulties (20 percent). Sixteen percent reported inadequate access to 
space, facilities, computers, or other resources, and 15 percent noted that they did not receive 
enough guidance from their host (or their host's research group). Less than 6 percent cited other 
difficulties related to their collaboration (e.g., insufficient credit for their contributions, being 
assigned a role less than merited by their skills or knowledge), or based on gender, disability status, 
or racial or ethnic, cultural, or religious backgrounds. 


Exhibit 4.7: Difficulties Experienced During the Fellowship 


Percent 

None 

39.7 

Logistical difficulties (e.g. transportation, navigating bureaucracy, etc.) 

31.5 

Communication or language difficulties 

20.2 

Inadequate access to space, facilities, equipment, computers, resources, supplies 

16.0 

Not enough guidance from host/host's research group 

14.5 

Not given credit for my contributions to advancing a project 

5.9 

Encountered barriers or discomfort based on my gender 

5.2 

Legal or medical difficulties in my host site 

4.9 

1 felt that my ideas were not treated with respect 

4.6 

1 was asked to do work that was someone else's responsibility 

3.7 

Encountered barriers or discomfort based on my cultural or religious background 

3.5 

My role on the project was less than merited my skills/knowledge 

2.9 

Encountered barriers or discomfort based on my race/ethnicity 

1.5 

Encountered barriers or discomfort based on a disability 

0.4 

Other difficulties 

9.7 


NOTES: Results do not sum to 100 percent because multiple choices were permitted. Missing Data: 8 Fellows. 
SOURCE: IRFP Applicant Survey-Item E5. 


Although fellows infrequently reported difficulties, the difficulties encountered varied considerably 
by region, thus responses are broken out by region in Exhibit 4.8. A greater percentage of fellows in 
East Asian locations experienced difficulties than fellows in other regions. The proportion of fellows 
who reported experiencing no difficulties ranged from 5 to 60 percent, with fellows placed in East 
Asian locations less likely than fellows placed in other regions to report no difficulties. The most 
frequent challenge fellows in East Asia reported were logistical difficulties (55 percent of such 
fellows). Such difficulties were also frequent among fellows residing in South or Central America (50 
percent) and Africa or the Middle East (46 percent). Fellows in East Asia were also more likely than 
other fellows to cite communication or language difficulties (45 percent) and inadequate access to 
space, facilities, equipment or resources (40 percent). In contrast, less than one-quarter of fellows 
experienced communication difficulties in Europe (24 percent), Africa and the Middle East (21 
percent). South and Central America (16 percent) or North America (12 percent). However, lack of 
access to facilities or resources posed a difficulty for non-trivial proportion of fellows in South and 
Central America (32 percent) and those in Africa and the Middle East (22 percent). Less than 15 
percent of fellows in Europe, the South Pacific, or North America encountered this problem. Finally, 
about twice the proportion of fellows in East Asia relative to fellows in other regions reported that 
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they had not received enough guidance from their host: 31 percent of fellows in East Asia cited this 
difficulty compared to 17 percent in North America, 15 percent in Africa and the Middle East, and 15 
percent in Europe. 

Thirteen percent of fellows in East Asian locations felt that their ideas were not greeted with 
respect. Notably, 10 percent of fellows in the South Pacific and North America each encountered 
barriers or discomfort based on their gender; such gender-based difficulties were also reported by 9 
percent of the fellows in East Asia and by those in Africa and the Middle East. 
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Exhibit 4.8: Difficulties Experienced During Fellowship, by Region of Host Institution 


Percent of Fellows Encountering Difficulties by Region 

East Asia 
(N=22) 

Europe 

(N=263) 

South & 
Central 
America 
(N=46) 

Africa & Middle 
East (N=29) 

North 

America 

(N=42) 

South 

Pacific 

(N=47) 

Logistical difficulties (e.g. transportation, navigating bureaucracy, 
etc.)* 

55.4 

28.4 

49.9 

46.2 

25.8 

16.7 

Communication or language difficulties* 

45.1 

23.7 

15.5 

20.8 

11.8 

0.0 

Inadequate access to 

space/facilities/equipment/computers/resources/supplies* 

40.3 

13.0 

32.3 

22.2 

12.1 

6.2 

Not enough guidance from host/host's research group 

30.6 

14.8 

10.7 

15.4 

17.2 

6.2 

Other 

17.7 

11.0 

4.0 

6.4 

11.5 

4.2 

1 felt that my ideas were not treated with respect 

12.8 

4.3 

4.3 

5.4 

7.0 

0.0 

Legal or medical difficulties in my host site 

8.6 

4.8 

4.3 

5.4 

2.2 

6.2 

Encountered barriers or discomfort based on my gender 

8.6 

2.8 

6.6 

8.6 

9.6 

10.2 

Not given credit for my contributions to advancing a project 

4.3 

6.4 

4.0 

9.0 

7.0 

2.0 

Encountered barriers or discomfort based on my race/ethnicity 

4.3 

1.7 

0.0 

0.0 

2.5 

0.0 

Encountered barriers or discomfort based on a disability 

4.3 

0.3 

0.0 

0.0 

0.0 

0.0 

My role on the project was less than merited my skills/knowledge 

0.0 

2.9 

2.3 

5.4 

4.8 

2.0 

1 was asked to do work that was someone else's responsibility 

0.0 

4.6 

0.0 

5.4 

7.0 

0.0 

Encountered barriers or discomfort based on my cultural or 
religious background 

0.0 

4.3 

2.0 

5.4 

2.2 

2.3 

None* 

4.9 

39.1 

35.7 

34.4 

47.5 

60.1 


*Regional differences were statistically significant at p <.05. 

EXHIBIT READS: 55.4 percent of fellows in an East Asian host location experienced logistical difficulties during their fellowship. 
NOTES: Results do not sum to 100 percent because multiple choices were permitted. Missing Data: 1 to 5 Fellows. 

SOURCE: IRFP applicant survey-item E5. 
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4.4.4 Challenges Cited by IRFP Hosts 

The most interesting finding related to hosts' difficulties is that the majority reported that they did 
not experience difficulty. Of those who reported any challenges in hosting an IRFP fellow (Exhibit 
4.9), the two most frequently cited difficulties encountered were the fellow not devoting enough 
time or effort to the research collaboration (10 percent) and concern that the fellow worked too 
independently and did not collaborate well with others (8 percent). 


Exhibit 4.9: Hosts' Reported Difficulties Hosting an IRFP Fellow 


Percent of 
Hosts (N=326) 

The fellow did not devote enough time/effort to the research collaboration 

9.8 

The fellow worked too independently, did not work well as collaborator/team member 

8.3 

Other difficulties 

6.4 

The fellow and 1 had differences of opinion about the direction of research 

4.6 

The fellow had unanticipated gaps in his/her preparation to conduct research with me 

4.6 

The fellow lacked sufficient understanding of cultural norms in my country 

3.4 

The fellow was disrespectful, caused conflict within my research group 

3.1 

The fellow needed too much guidance 

1.8 

The fellow's lack of familiarity with the primary language spoken in my research group 

1.2 

made collaboration more difficult than anticipated 


None of the difficulties listed 

65.3 


Missing Data: 2 Hosts. 

SOURCE: IRFP Host Survey-Item E3. 

4.4.5 Fellows’ Satisfaction with IRFP 

Both fellows and hosts were, on the whole, satisfied with their collaboration (Exhibit 4.10). Over 80 
percent of fellows were very or somewhat satisfied with the match between their own and their 
host's research interests, with their host's expertise and contributions to the research, with hosts' 
inclusion of fellows in meaningful collaboration and efforts to help fellows meet other researchers, 
and with the frequency of meeting with their host. However, about one-fifth (21 percent) of fellows 
were somewhat or very dissatisfied with the guidance or mentoring provided by their host. Fellows' 
open-ended responses provide some insight. Among 359 open-ended comments on areas of 
satisfaction and/or dissatisfaction with their IRFP experience, about 10 percent expressed difficulty 
interacting with their host or other colleagues at the host institution. For example: 

My mentor was close to retiring and was not fully engaged with the day to day workings in 
the laboratory. The project ... that he presented to me when I was applying to IRFP was not 
really taking place when I arrived. I had to start from scratch without really very much 
laboratory support. (IRFP fellow) 

The host was rarely present and attempted to run the laboratory via telephone or email. 
Moreover, there was no gualified laboratory scientific management or personnel to help run 
the lab in his absence. (IRFP fellow) 


Abt Associates Inc. 


4. Fellowship Experience 47 



Evaluation of International Research Fellowship Program: Final Report 


Exhibit 4.10: Fellows' Satisfaction with Host 


Host's expertise in his/her field 


Match between yours and your hosts' 
research interests 

Host's inclusion of you in research 
group/laboratory, 
meaningful collaboration 

Frequency of meetings with host 


Host's efforts to help you meet other 
researchers 


Level of host's intellectual contribution 
to your joint research project 


Guidance or mentoring provided by host 



■ Very Satisfied 


Somewhat satisfied 


■ Somewhat dissatisfied Very dissatisfied 


N=457, Missing Data: 11 to 13 Awarded Applicants; Guidance or Mentoring Provided by Host: 18 Awarded Applicants. 
SOURCE: IRFP Applicant Survey-Item E7. 


Ninety-two percent of fellows were somewhat or very satisfied with the quality of research they 
were able to conduct as part of IRFP. Over 90 percent of fellows were somewhat or very satisfied 
with several other aspects of their IRFP fellowship (Exhibit 4.11, top and bottom panels), including 
the opportunities to learn about their host country (99 percent), and the timing of the fellowship 
with respect to their career goals (97 percent). More than 92 percent were somewhat or very 
satisfied with the amount of fellowship support. Though few fellows indicated dissatisfaction, the 
most common areas of dissatisfaction were the duration of the fellowship (13 percent) and the 
research facilities and access to the internet at their host institution (10 percent and 11 percent, 
respectively). 
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Exhibit 4.11: Fellows' Satisfaction with Aspects of IRFP Fellowship 


0 20 40 60 80 100 


Opportunities to experience/learn 
about 

culture, history, geography, etc. in 
host site 

The quality of the research I was able 
to conduct 


Fellowship timing with respect to 
your career 







87.4 11.8 0. 

- 





62.4 

29.6 5.6 

- 





84.0 12.9 1.5 








■ Very Satisfied Somewhat satisfied 

■ Somewhat dissatisfied Very dissatisfied 


N=457, Missing Data: 9 to 12 Awarded Applicants (top and bottom panels). 
SOURCE: IRFP Applicant Survey-Item E7. 


4.4.6 Hosts’ Satisfaction with IRFP 

Similarly, hosts had positive perceptions of their fellows and their experience hosting their IRFP 
postdoctoral fellow (Exhibit 4.12). Hosts strongly agreed (72 percent) or agreed (24 percent) that 
their IRFP fellow had sufficient knowledge and expertise, and strongly agreed (66 percent) or agreed 
(26 percent) that the fellow integrated well with their research group. Most hosts felt that fellows 
balanced appropriate caution (41 percent strongly agreed and 48 percent agreed) and with 
appropriate risk-taking (37 percent strongly agreed and 46 percent agreed) in conducting research. 
The only area of concern for some hosts was the duration of the fellowship: more than 40 percent of 
hosts agreed or strongly agreed that the duration was too short. 
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Exhibit 4.12: Hosts' Perception of the Fellow and Their Collaboration 


0 


The fellow had sufficient knowledge and expertise 
to fully participate in a research collaboration with 
me 

The fellow integrated with staff/ 
members of my research group 

Scientific cultures of our countries are similar, 
making productive collaboration possible 

The fellow spent sufficient time working on the 
project 

The fellow exercised appropriate caution 
in his/her approach to research 

The fellow was willing to take appropriate risks 
necessary for research 

The amount of time for the fellowship 
experience was too short 

Language was a barrier to Fellow's ability to interact 
with me/my research group 

Lifestyles and general cultures of our countries are 
a barrier to collaboration 

■ Strongly Agree Agree ■ Disagree Strongly Disagree ■ Not applicable 

N=328, Missing Data: 2 to 6 Hosts. 

SOURCE: IRFP Host Survey-Item C2. 

IRFP fellows also compared favorably to other postdoctoral fellows with whom hosts had worked, as 
83 percent were at least equally satisfied with their IRFP fellow compared to other postdoctoral 
fellows (see Exhibit 4.13). About 10 percent were somewhat less satisfied, and 7 percent were much 
less satisfied with their IRFP fellows. In open-ended comments, hosts specifically cited IRFP fellows' 
scientific knowledge, high levels of motivation and commitment, and the ease of interacting with 
the fellow: 

[The fellow] led a large experimental study still ongoing that represented one of the most 
ambitious and interesting experiments ever conducted in my lab. This study also brought in other 
collaborators (students and Pis) and represents one of the most satisfying collaborations of my 
career. (IRFP host) 
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Hosting [the fellow] was a pleasure. He was a highly motivated researcher and very productive. I 
fell that both he and I (and [the institution]) benefitted from our collaboration. I was not aware 
that the IRFP program was still ongoing but I will take steps to encourage more U.S. postdoctoral 
fellows to apply to it. (IRFP host) 

The fellow is a hard working and very enthusiastic researcher, well trained at U.S. during her 
PhD. She integrated very easily our team and will be a future collaborator. (IRFP host) 

It was always a pleasure to have this particular fellow around. He was ... a sociable and modest 
person whilst being a creative and industrious researcher. A very charming combination indeed. 
(IRFP host) 


Exhibit 4.13: Hosts' Satisfaction with IRFP Fellow Compared to Other Postdoctoral Fellows 


How satisfied were you overall with this IRFP postdoctoral Fellow compared 
to other postdoctoral fellows? 

Percent of Hosts 
(N=307) 

Much more satisfied 

28.3 

Somewhat more satisfied 

27.4 

Equally satisfied 

27.7 

Somewhat less satisfied 

9.8 

Much less satisfied 

6.8 


NOTES: Only those hosts who had worked with postdoctoral fellows other than the IRFP fellow were included (N=307, 
Missing=21)). 

SOURCE: IRFP Host Survey-Item El. 

Although the majority of hosts held positive views of the fellow and of the IRFP program as a whole, 
a handful of host scientists expressed dissatisfaction with the program or with their particular 
fellow. Of the 279 hosts who commented on what was most satisfying or unsatisfying about the 
fellow or the IRFP fellowship program (Item Ela), only about 5 percent had expected a greater 
number of joint publications or a more fruitful subsequent collaboration. Five percent mentioned 
interpersonal problems with the fellow. One host, for example, noted that "this [IRFP] postdoc did 
not focus enough on the project we had together, and tried to keep several collaborations 
elsewhere, travelling abroad regularly." Although rare, a handful of hosts felt that the IRFP program 
was too short: 

We spent most of the year laying foundations and if we had continued for a second year we 
would have produced many more outputs. (IRFP host) 

... the IRFP financing was shorter than the natural duration of the project, and thus the fellow 
could not stay at my institution until the logical conclusion of the research (which required a year 
longer than the IRFP financing). (IRFP host) 

4.4.7 Fellows and Hosts Would Recommend IRFP to Others 

I recommend [IRFP] most highly to every grad students who comes through my office (I have also 
led informal postdoc workshops where I strongly encourage students to apply). The opportunity 
to spend an extended amount of time dedicated to research in a different cultural setting ...was 
an incredible experience. I moved on to the job market with renewed energy, new perspectives 
on my research, and a greater appreciation for international collaboration (and a stronger C V). 
(IRFP fellow) 
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Former fellows and hosts overwhelmingly reported that they would recommend (or had already 
recommended) participation in the IRFP program to others. One hundred percent of fellows 
endorsed the IRFP program, and nearly 80 percent would recommend their former host scientist 
(Exhibit 4.14). 


Exhibit 4.14: IRFP Fellows' Willingness to Recommend IRFP, Host Country and Host 
Scientist to others 


1 Would you recommend... 

Percent of Fellows 

... IRFP to a colleague? 

Yes 

100 

No 

0.0 

... your host country to a colleague? 

Yes 

97.3 

No 

2.7 

... your host scientist to a colleague? 

Yes 

79.4 

No 

20.6 


NOTES: Only IRFP fellows who had completed their fellowship by October 1, 2010 (N=384) received these 
items on the survey. Missing Data: 8 to 10 Fellows. 

SOURCE: IRFP Applicant Survey-Items F9-11. 


Former hosts also exhibited strong enthusiasm for the IRFP program. As shown in Exhibit 4.15, 84 
percent of hosts would unequivocally recommend hosting an IRFP fellow to others. Another 14 
percent said that their recommendation would depend on the qualifications of an individual IRFP 
candidate. Less than 2 percent indicated any hesitation to recommend the program. 


Exhibit 4.15: Hosts' Willingness to Recommend IRFP to Others 


Would you recommend (or have you recommended) to others that they host 
an IRFP postdoctoral fellow from the United States? 

Percent of Hosts 
(N=321) 

Yes 

84.1 

No, 1 would not recommend to others that they host an IRFP postdoctoral 
fellow from the U.S. 

0.6 

1 might recommend hosting an IRFP fellow, but it depends on the qualifications 
of the individual postdoctoral candidate 

14.0 

1 might recommend hosting a postdoctoral fellow from the U.S., but the IRFP 
program created challenges for me or my postdoctoral fellow 

0.9 

1 am not sure 

0.3 


Missing Data: 7 Hosts. 

Sources: IRFP Host Survey-Item E7. 


More than one-third of hosts who would recommend IRFP commented on the high quality of the 
fellows: 

These are dedicated scientists with excellent background [sic]. They can work independently, but 
are willing to collaborate. (IRFP host) 

About one in five hosts cited the cross-cultural benefits of hosting an IRFP postdoc: 

The whole lab benefited from the experience. We were enriched by new theoretical/analytical 
perspectives. We were enriched by new ways to deal with data collection, data analysis, lab 
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meeting dynamics and research in general. The lab in general was stimulated to search for 
collaboration with researchers abroad. (IRFP host) 


4.5 Descriptions of Experiences 


Both fellows and hosts elaborated on their experiences with the program through open-ended 
responses. 

4.5.1 Fellows 

Fellows were given an opportunity to elaborate on the most positive aspects of their IRFP 
experience, and 425 respondents provided a response. The large majority elaborated on 
professional experiences and benefits, and about 20 percent described a personal experience. 
Ninety-three people mentioned a personal experience (e.g., the opportunity to be immersed in a 
new culture, improved language skills, the development of long-lasting friendships, etc.). 

I made great friends and made good memories. I was able to become a part of a local 
community and to really experience things as something other than an outsider. After three 
months, I was thinking and speaking in the language. I joined a local sports club with which I 
skied in the winter and kayaked in the summer. This club had a local meeting place / sports 
facility where we would gather 1-3 evenings per week. I came away from the IRFP experience 
having been genuinely immersed in the culture. Although I would never be happy settling in that 
country, I came away with a fondness for the place. (IRFP fellow) 

Over one hundred fellows (115) commented that the most positive aspect of the fellowship was its 
provision of unique opportunities and experiences that they would not have received elsewhere. 
Some discussed how these experiences paved the way for research opportunities after the IRFP 
fellowship had ended, while others described new skills that they acquired as a result of their 
experience. 

I was able to pursue research guestions that I was extremely interested in and could not have 
pursued in the US. This experience has helped define my research direction for many years to 
come. (IRFP fellow) 

For me, the most positive aspect of my IRFP experience was the chance to work with [country] 
scientists and conservationists during the 1990s, an exciting period of change when the 
involvement [country] science infield ecology and conservation was just at its initial stages. 
Together, my colleagues and I made exciting discoveries and stimulated the interest of others in 
[country] about biodiversity and the need to protect the natural world. (IRFP fellow) 

The most positive aspect of the IRFP experience was acting as a PI. I formulated the research 
plan, wrote the proposal, was rejected the first year I applied, took the criticisms of the reviewers 
and re-wrote the proposal, won the second year I applied, went to the host institution, set up the 
laboratory, conducted research independent, wrote the paper and published. As an overall 
experience and a mini-version of what I do now as an academic, it was a great experience. (IRFP 
fellow) 
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It allowed me to follow an interesting direction for my research that was possible only by 
working in collaboration with the host institution and conducting fieldwork in the host country. I 
chose this country specifically because regional biogeography allowed me to follow the path of 
inquiry that I had hoped to examine because the various taxa that I wanted to study come into 
contact exclusively in this region. (IRFP fellow) 

The opportunity to develop and implement a new research program. Working largely 
independently— but with a very experienced and supportive host readily available— was an ideal 
stepping stone between graduate school and a faculty position. (IRFP fellow) 

One hundred and nine respondents described how the program allowed them to develop and 
strengthen their professional networks by introducing them to researchers and colleagues they 
might not have otherwise been able to meet. 

... by far the most positive aspect of my IRFP experience was that I met astronomers in [country] 
that I might not have otherwise gotten to meet, and this has led to multiple collaborations. (IRFP 
fellow) 

It allowed me to collaborate on a scale uncommon to most U.S. labs and has led to most of my 
success where all of my research is interdisciplinary. (IRFP fellow) 

I really met a lot of people who helped me in one way or another that year and later. Knowing 
the researchers from my host institute helped me to recruit newly graduated students to help 
with my job. It also connected me with researchers from Australia who have continued to 
collaborate to this day. (IRFP fellow) 

Fifty-two people elaborated on the positive effect of the IFRP program on their career. Some 
became more aware of potential career choices and refined their future goals, while others noted 
that the experience bolstered their CV and made them more competitive for subsequent positions. 

... it shaped my entire research program from 1998 to now and into the future. It has been the 
most important event in my career. (IRFP fellow) 

It was my first chance to completely direct my own research, since I brought my project idea to 
the host lab. Also, my gaining more comfort with the country and language— and gaining 
familiarity with research there— contributed to my accepting a permanent position in that 
country years later. (IRFP fellow) 

The most positive aspect is that I benefited greatly as a scientist. I now have a faculty position (in 
the U.S.), and I think my IRFP experience played a large role in me winning the opportunity. (IRFP 
fellow) 

Finally, 30 people felt that a very positive aspect of the program was that it exposed them to 
different research cultures and new perspectives. 

I was able to spend a significant amount of time (several years) in a foreign country, which has 
expanded my understanding of the U.S. In particular, I experienced a very different research 
environment that I will try to replicate in my own laboratory. (IRFP fellow) 
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The fellowship allowed me to understand how research is undertaken at [university], and to 
interact with the best researchers in my field. It also allowed me to travel to Spain and work in 
several Spanish Universities, where the research and academic structure are much different than 
that in the U.S. These contrasts among countries, in research and teaching approaches, were 
what I most appreciated about my fellowship. (IRFP fellow) 

4.5.2 Hosts 

Hosts elaborated on what they perceived to be the best aspects of their affiliation with IRFP. There 
were 261 responses to this question. Hosts cited specifics of the fellows, the research, and the 
products resulting from the collaborations. 

Half of the hosts (133) commented that some of the best aspects were the outcomes that resulted 
from these collaborations. Specifically, 108 of the hosts noted how they or their lab experienced 
some type of advancement in their research (e.g., new lines of research opened up, they became 
aware of new research questions, interests, or techniques, etc.) as a result of IRFP. 

The IRFP fellow came to work in a subject [discipline]) in which none of my group had worked 
before, even though we had developed tools that could be applied in this field. Fie therefore gave 
us the possibility of doing work in a new field, which is not strongly developed in [location], (IRFP 
host) 

The fellow made significant advances that greatly improved our understanding of our research 
topic. He was full of good ideas and several led to fruitful projects and excellent publications. 
(IRFP host) 

Collaboration was great and we have identified a very interesting area of collaboration. 
Interaction with the fellow in small meetings and in lab meetings was great. She has now a very 
strong and interesting database with which we will start publishing and will develop new 
collaborations. It was interesting to see the different approaches to many logistic aspects of 
research and to learn from these differences. The Postdoctoral fellow stimulated my students to 
think in different directions, to try different statistical analyses. She helped them with their 
English writing. (IRFP host) 

His knowledge and ideas enhanced the intellectual atmosphere of our research group. He 
obtained very good data and they yielded interesting conclusions and led to further work. My 
continued contacts with him (discussions of guestions of mutual interest) have been valuable. 
(IRFP host) 

[The fellow] contributed to the intellectual life in my laboratory by interacting with my graduate 
students, demonstrating laboratory technigues to them, and providing a broader perspective on 
the research field. The work we did was successful although it still has not reached the 
publication stage. (IRFP host) 

There was an excellent confluence between the group's direction and the background of the 
fellow, which resulted in a scientifically productive project which benefited all. (IRFP host) 
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[The fellow] was very productive as a researcher and as a result a number of papers got 
published that probably would not have been achieved without her substantial input. She was 
also easy to work with, and very highly motivated. (IRFP host) 

Ninety-one hosts noted that the best aspect was the resulting collaboration that occurred, either 
with the fellow, the fellow's U.S. institution, or other international institutions. 

[The fellow] is indeed an independent thinker willing to take risky approaches. He often 
challenged my ideas which often resulted in very productive discussions. (IRFP host) 

This collaboration was extremely fruitful, and still continues. We have projects, and 1 student, in 
common. (IRFP host) 

The increased cooperation with the fellow's home institution and his colleagues/former advisors 
and the special research expertise he brought to the particular project at hand. (IRFP host) 

That I established a long-term collaborative link with a U.S. researcher, who was able to 
establish himself as a new faculty member in Ohio, making it possible to build a long-term link 
between our labs. (IRFP host) 

Forty-one hosts reported that the best aspect of the program was having the opportunity to work 
with a fellow who displayed impressive academic knowledge and skills, and 33 hosts were 
impressed with the fellow's interpersonal skills, and their ability to successfully integrate into the 
lab. 


The best aspects was that [the fellow] brought valuable complementary competence in 
molecular analyses (and sexual selection) to our lab, and helped train students in my lab and also 
beyond (he organized a highly rated PhD course in such methods for students from all Nordic 
countries, after having successfully applied for funding of the course to the Nordic Marine 
Academy). (IRFP host) 

[The fellow] was ... a very stimulating scientist, eager to collaborate, and open for scientific 
discussions. He operated at the frontiers of fundamental research in my group and was a 
pleasant and very cooperative colleague. His ambition is a tenure-track position at a U.S. 
university, which he has acquired and which I fully support. (IRFP host) 

The fellow, besides being a good scientist, also had excellent social skills, and he played an 
important role in improving the relations between people in our group and the overall 
atmosphere in our group. (IRFP host) 

As noted previously, this was the start of a long term collaboration extending well beyond the 
original project. The individual is a talented academic, a great experimentalist, open to new 
ideas and a real pleasure to work with. (IRFP host). 

[The fellow] was a good citizen who helped others with their own research problems, as well as 
improving the social life of the group (IRFP host). 
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He worked extremely hard and used his complete time frame for performing research. This 
included seven days a week including holidays. Additional he made nice social effort to approach 
my family. And he did not complain about the restrictions to work scientifically in the culture of 
[location], (IRFP host) 

Excellent scientific discussions. He was always very helpful, with a good academic formation and, 
in general it was very pleasant to discuss data analysis and science in general. (IRFP host) 

He was a considerable stimulus to our research and interests, and I believe that he similarly 
benefited from his time here. He was one of about 6 postdoctoral researchers in my group at 
that time, and he contributed fully to the group and was a great asset. (IRFP host) 

Although we have not collaborated closely in the last few years, I and members of my group 
have kept in touch with [the fellow]. We have involved her in Special Sessions at conferences. 
While she was in the lab, she was very active in group discussions and in encouraging other 
researchers. (IRFP host) 

Eleven hosts reported that the best aspect was the financial support they received from I RFP. Five 
noted that the best aspect was participating in the fellow's professional growth process. 

It [the program] allowed a U.S. fellow to join my laboratory and provided some research and 
travel money for the fellow, which many fellowship programs do not. This is an ideal system to 
foster collaborations with U.S. scientists. (IRFP host) 

My postdoc came to my lab to learn about investigating the mechanical properties of 
biomaterials, and it was really a pleasure to see him grow in skill, knowledge and enthusiasm in 
his research. (IRFP host) 

Thirty-one responses described other aspects (e.g., co-supervised and trained younger students, 
became personal friends, etc.), and 15 hosts indicated that there was either no best aspect of the 
program, or they provided information on a negative experience. 


.6 Recommendations from Former Fellows and Hosts 


[IRFP] is a very valuable program in the NSF portfolio, helping to develop U.S. researchers and 
foster new multi-national collaboration. In difficult economic circumstances it is easy to look for 
savings in areas such as this, ...but i would say that in helping to develop more rounded young 
scientists this program returns a great deal in benefits to the intellectual capital of the U.S. for a 
relatively modest outlay. (IRFP host) 

Responses to open-ended request for recommendations for the program from fellows and hosts 
provide some insight into areas that might benefit from improvements. First, although 279 fellows 
provided a response to this question, 77 respondents commented that they would not change 
anything about the program, leaving only 202 who made an actual recommendation. The 
recommendations for the program made by fellows and hosts align with addressing some of the 
challenges that were reported previously in this chapter. 
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4.6.1 NSF’s Timing and Flexibility in Award Administration Is Appreciated 

As presented in Chapter 3, the majority of applicants received notification from NSF in sufficient 
time to make arrangements (87 percent). Those who received an IRFP award were more likely to 
find the timing of notification appropriate (94 percent) than were unfunded applicants (82 percent). 
Several former fellows appreciated the flexibility NSF provided when the timing of their IRFP award 
conflicted with other opportunities or events. For example: 

I was awarded the IRFP at about the same time I received my first tenure track faculty position 
offer. The fact that the program allowed me to modify my plans so that I could accept the faculty 
position and still conduct the research proposed in the IRFP (by using the IRFP to ... travel to the 
foreign country over several summers) was very important to me. I would not have developed an 
international research program if the IRFP had not allowed this flexibility. (IRFP fellow) 

I had a baby just prior to being notified that I had received the fellowship, and I really 
appreciated the opportunity to extend the research funds portion of the grant without penalty so 
that I can work part-time while my [child] is still young without compromising my ability to 
complete the research project. (IRFP fellow) 

Flowever, one former fellow noted that the timing of award notification was "a little off." The fellow 
reported, 

I graduated in May, and didn't get the fellowship until the beginning of the next year. It was 
difficult to figure out if I was going to have funding to do the post-doc abroad. It would be nice if 
there was a way to apply twice a year. (IRFP fellow) 

4.6.2 The Duration of the IRFP Fellowship Should Be Lengthened 

Of the 202 fellows who suggested changes to the IRFP program, 60 recommended increasing the 
length of the fellowship; this recommendation came from fellows who had been awarded one- or 
two-year fellowships. 

Principally, because of the length of the fellowship, most of what I did was move. Just when I was 
getting settled in figuring out my host institution and making progress on the project, I had to 
leave to start all over again for the domestic component. And then during the domestic 
component I had to divide my time between the first part of the project, the second part of the 
project and finding a job because the fellowship was ending. A three year grant would 
ameliorate some of these difficulties. (IRFP fellow) 

Some host scientists also shared this view. One host was particularly dismayed by the short 
duration: 

It lasted one year only, which is not sufficient for the type of project involved. We would have 
been far more productive if she had been here for 2 years. Returning to the US. just scuppered 
what we were doing and caused her to have to start other projects. A real shame for everyone 
involved. Flexibility was absent. (IRFP host) 
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4.6.3 IRFP Fellows Would Benefit from Advice Regarding Financial and Logistical 
Challenges 

Almost 12 percent of those fellows who described their most challenging experience during the IRFP 
fellowship mentioned unexpected financial difficulties due to fluctuating currency exchange rates. 

The IRFP salary allowance does not allow for adjustments in exchange rate. My salary was fixed 
... when the exchange rate between the U.S. and [the currency of my host country] was 
favorable. In six months the exchange rate flip-flopped and my working salary was reduced by 
30%. This created a very difficult economic scenario with regards [sic] to rent and food and basic 
living expenses. Having to ration created a stressful experience. (IRFP fellow) 

The biggest challenge I faced was in handling the accounting and finances of this grant. The 
transition from being a graduate student, and receiving relatively small research support with an 
accounting structure in place within the doctoral university, to suddenly receiving a multi-year 
grant with the money coming directly to the fellow was overwhelming. I am happy with the 
flexibility that came with this fellowship, ... but I think NSF should provide more help with at 
minimum guestions about foreign income and tax status. (IRFP fellow) 

The most challenging aspect [of IRFP] was dealing with the bureaucracy that comes with 
working in a foreign country, such as setting up a bank account, driving license, visa, and bank 
transfers from the U.S. (the fellowship money is paid in U.S. currency to a U.S. account and the 
exchange rate changed dramatically during the course of my fellowship, not in my favor. (IRFP 
fellow) 

I also found it very frustrating to pay for things using a U.S. bank account. I had not taken into 
account the cost of transferring funds to [my host country] in my budget. (IRFP fellow) 

Recommendations to address the financial issues included providing information to fellows about 
how the fellowship funding is distributed and cautioning potential fellows about the possible 
consequences of unfavorable exchange rates; allowing applicants to propose a higher budget if the 
cost of living in a proposed host location is exceptionally high; and setting up a support network, 
perhaps comprised of former fellows, to provide information on the financial, logistical burdens of 
working and living abroad: 

When I was [a fellow], there was no network to support any of us— we were scattered all over 
the world, and there weren't any forums to ask guestions about how to deal with issues like 
health insurance, immigration visas, etc. specific to a given country. (IRFP fellow) 

A handful of former fellows recommended that an alumni gathering of former IRFP fellows could 
also provide a forum for newly awarded fellows (or even IRFP applicants) to exchange helpful 
information: 

The IRFP program would benefit from more active internal (IRFP) networking activities (for 
example, alumni gatherings or even a mini-workshop bringing current and past fellows in 
contact). (IRFP fellow) 
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4.6.4 Advance Language Preparation Would be Beneficial 

Both former fellows and hosts suggested that IRFP fellows would benefit from better language 
preparation in advance, even in countries where most citizens speak English: 

Learning a new language is very difficult. Luckily, the scientists all spoke English, but it was 
sometimes frustrating to not be able to participate fully in conversations. (IRFP fellow) 

Try to learn a bit of [the language]. It encourages ... people to make an effort and speak English 
with you (They also usually like to help you speak [the host country's language] better.... It is 
important for the social life. (IRFP host). 

In countries where English is not widely spoken, the importance of advance preparation may be 
more critical. As one fellow noted, 

I studied [the host country's language] part-time when I first arrived, but struggled with the 
language as most foreigners do. By the end of my time in [my host country] I became very 
frustrated by my mediocre ability, which I felt prevented me from assimilating better. (Former 
IRFP fellow 

In particular, the language difficulties experienced by a substantial proportion of fellows in East 
Asian locations suggest that those intending to pursue an international collaboration in this region 
might benefit from more intensive language preparation. 

4.6.5 Fellows Should Discuss in Advance the Availability of Institutional Resources and the 
Host’s Support 

Addressing the lack of adequate access to facilities or resources poses a more difficult challenge, 
since there is less likelihood that a postdoctoral fellow can prepare for this in advance. Fellows 
might be advised to discuss with their host in detail the facilities, equipment, and other resources to 
which they will and will not have access during their fellowship, what resources the host institution 
has to maintain, repair or replace equipment that may break during the fellowship, and what 
arrangements fellows may need to make in advance of their visit (e.g., submitting paperwork to 
arrange identification allowing access to research facilities, or secure time on shared equipment). 

A handful of fellows indicated that their intended hosts were largely absent from the laboratory or 
research facility during their stay; two mentioned that the host departed from the host institution 
before or shortly after the fellow arrived, despite having agreed to collaborate with the fellow. 

About one in ten hosts who offered recommendations for U.S. scientists considering an 
international postdoctoral fellowship recommended some advance preparation to become familiar 
with the host institution and potential host scientist. One such host emphasized: 

It is very important to prepare the ground by establishing contact beforehand, find out what the 
most interesting research interests might be, given the local expertise, available data and 
resources. In our case we did this partly by preparatory workshops (supported by a grant from 
NSF) and also by invitations to other conferences or workshops (that we hosted). (IRFP Flost) 
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4.6.6 Benefits of IRFP Collaborations 

Several former fellows and hosts mentioned the importance of cross-cultural exchange to their 
professional and personal development. For example, many former fellows cited the benefits of 
international collaboration for broadening their research skills. As one former fellow put it, 
"scientists in different countries have very different ways of approaching a problem. Through IRFP 
my technical and philosophical toolbox has been increased a lot." Another indicated that she 
"learned new research and analytic skills— techniques that, at the time of my fellowship, were not 
widely being used in the USA." 

Other fellows credited their international collaboration with having a transformative effect on their 
professional development: 

My first field trip with researchers in my field that were from my IRFP host institution 
[was one of the most memorable events]. It was immediately apparent that the methods 
and techniques I learned at my U.S. graduate institution were not the only way of doing 
things! This really taught me to think critically about how to best conduct research and 
the value of thinking laterally about problems/questions and how this can yield very 
interesting results. That exposure to a different "worldview" has been invaluable and 
continues to serve me very well indeed. (IRFP fellow) 

IRFP was the most enlightening professional experience I have ever had. Getting the 
opportunity to work with researchers outside the U.S. has opened up a whole new world 
of ideas and ways of doing things, and has made me a much more confident, effective, 
and innovative researcher. (IRFP fellow) 

Many hosts also indicated that they had benefitted from the cross-cultural exchange: 

The IRFP fellow came to work in a subject ... in which none of my group had worked 
before, even though we had developed tools that could be applied in this field. Fie 
therefore gave us the possibility of doing work in a new field, which is not strongly 
developed in [location], (IRFP host) 

The whole lab benefited from the experience. We were enriched by new 
theoretical/analytical perspectives. We were enriched by new ways to deal with data 
collection, data analysis, lab meeting dynamics and research in general. The lab in 
general was stimulated to search for collaboration with researchers abroad. (IRFP host) 

These benefits and other outcomes of the program are systematically explored in the next chapter. 


Abt Associates Inc. 


4. Fellowship Experience 61 


Evaluation of International Research Fellowship Program: Final Report 


5 Outcomes of Participation in IRFP 


My time as an IRFP [fellow] was transformative both personally and professionally. It was a 
terrific experience whose benefits I continue to reap even 10 years later in my current 
position. I met wonderful colleagues with whom I still collaborate, and I feel as though [my 
former host country] is my second research home although I was only therefor just over a 
year. I have been back numerous times, most recently with students from my home 
institution.... I'm very grateful for having been awarded an IRFP [fellowship], and have 
recommended the program to younger colleagues. (IRFP fellow) 

The IRFP program is designed to provide new doctoral recipients with experiences that will have 
lasting effects on their careers, and by extension, on a broader network of colleagues, associates, 
and U.S. institutions. Two key objectives of the IRFP program are to further IRFP fellows' global 
perspective in their field and to enable fellows to forge long-term, sustained relationships with 
colleagues abroad. 54 Further, IRFP fellows may assume leadership roles in promoting the benefits of 
international collaboration to younger generations of scientists and engineers. 55 

This chapter describes impacts of IRFP by examining whether and how the international 
engagement and career trajectories of former IRFP fellows differed from those of their unfunded 
peers (matched using propensity-score methods detailed in Appendix C). Two sets of outcomes are 
tested. The first set of outcomes, relating to fellows' subsequent engagement in international 
collaboration, is directly related to the primary goals of the IRFP program. The second set of 
outcomes, relating to fellows' subsequent career paths, was examined to understand whether the 
time spent abroad is detrimental to the later careers of early career scientists and engineers. 

The impact analyses are then followed by comparisons of IRFP applicants to a nationally 
representative sample of STEM doctorate recipients to provide a context in which to understand the 
outcomes of IRFP fellows and applicants. Finally, the chapter concludes by describing former IRFP 
fellows' and hosts' reflections on the outcomes of the fellowship. 

Specifically, this chapter answers the following questions: 

• Does the extent to which former fellows engage in international collaborations differ from 
those of unfunded applicants? 

• Do fellows' post-award career activities and job characteristics differ from unfunded 
applicants and other STEM graduates in the U.S.? 

• What do program participants view as the outcomes of the program? 

• Do the outcomes of program participation extend beyond the direct participants? 


NSF. 2006a. 

NSF. 2005. IRFP Program Solicitation. Retrieved from 
http://www.nsf.gov/pubs/2005/nsf05599/nsf05599.txt 
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5.1 Key Findings 


• There are statistically significant and positive differences between fellows and unfunded 
applicants on several facets of international research, including: number of international 
postdoctoral fellowships; number of publications with a foreign co-author; and percentage 
of publications with a foreign co-author. The fellows' research productivity is consistently 
higher across all three measures. 

• The IRFP fellows' engagement in international research collaborations has not been 
detrimental to their career opportunities or professional advancement in the U.S: in fact, 
fellows and their peers were equally likely to hold multiple postdoctoral appointments, and 
were equally productive researchers, equally likely to hold a faculty rank of assistant, 
associate, or full professor, and equally likely to be tenured. 

• Career outcomes of IRFP fellows, and applicants overall, compare well against national 
STEM PhD holders on employment, publications, and international collaborations, 
suggesting that IRFP attracts a talented pool of applicants. 

• About four-fifths of fellows (79 percent) reported that participating in IRFP had qualified 
them for a broader range of career options, and 68 percent said that IRFP had made them 
more competitive for jobs. 

• Nearly three-quarters (71 percent) reported that they had made important connections to 
researchers in their host country, and had opened up new areas for investigation, and about 
two-thirds commented that their IRFP fellowship provided the chance to familiarize 
themselves with the scientific enterprise in their host site and to make substantial 
advancements in their research (65 and 64 percent, respectively). 

• Most fellows reported that the fellowships offer opportunities for professional relationships 
that endure beyond the fellowship period, either through subsequent collaborations with 
their hosts, and/or additional communications (46 percent each). Half of all former fellows 
said that participation in IRFP made them more committed to international research 
collaboration. 

• More than three-quarters of former IRFP fellows reported that post-fellowship, they taught 
colleagues, students or peers methods learned during their fellowship (78 percent), and 
shared resources or tools acquired during this time (75 percent). 

• Flosts also served as a mechanism for extending the effects of the program to other 
scientists. Twenty-five percent of former IRFP hosts reported that a collaboration with a U.S. 
researcher resulted from their participation in the IRFP program. 


5.2 Comparison of IRFP Fellows and Unfunded IRFP Applicants 


The impact of the IRFP program on its participants was examined by comparing IRFP fellows to a 
rigorously matched group of unfunded applicants, controlling for number of years since PhD degree, 
underrepresented minority status, and gender, and also where applicable number of pre-award 
publications and field of study. Impact models used the propensity-score matching methods 
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detailed in Appendix C. The overall treatment effect was calculated by taking an average of the 
estimated treatment effects weighted by the number of treated observations (i.e., the number of 
awardees) within each stratum. Exhibits display the adjusted (estimated) means for awardees and 
non-awardees. What emerged are insights into the unique effects of IRFP program. Two sets of 
outcomes were tested: first, a set of outcomes relating to fellows' subsequent engagement in 
international collaboration, reflecting the primary goals of the IRFP program; and a second set 
relating more generally to fellows' career paths. 56 

5.2.1 International Collaborations 

The IRFP program is meeting its primary goals to facilitate productive international research 
collaborations among its awardees. As summarized in Exhibit 5.1, compared to a matched group of 
unfunded IRPF applicants, IRFP fellows on average were more likely to establish productive 
international research collaborations. Specifically, statistically significant differences emerged, 
whereby fellows held a greater number of international postdoctoral fellowships; 57 produced a 
greater number of publications with a foreign co-author; and produced a higher percentage of 
publications with a foreign co-author. 

IRFP fellows had held an average of 1.1 international postdoctoral fellowships, whereas unfunded 
IRFP applicants had held an average of 0.63 international postdocs. 58 That the average number for 
fellows is close to one suggests that the IRFP was likely the only international postdoc they had ever 
held. The non-zero average for unfunded applicants means that some unfunded applicants did go on 
(after an unsuccessful application to IRFP) to hold at least one international postdoc. Flowever, the 
statistically significant difference in the number of international postdocs reveals that some 
proportion of unfunded applicants never went on to a different (non-IRFP) international postdoc. 

On average, IRFP awardees produced 12.8 publications with a foreign co-author, 6 more 
publications than unfunded applicants. Moreover, a statistically significantly higher percentage of 
IRFP awardees' publications were internationally coauthored than were those of unfunded 
applicants (37 and 27 percent, respectively). In terms of the number and rate of internationally co- 
authored publications, the IRFP program facilitates productive international research collaborations. 


The estimates for former IRFP Fellows compared to SDR respondents differ from the IRFP impact 
estimates shown in this section: in the comparison to SDR sample, covariates were grand mean centered 
using the average mean of the SDR and IRFP applicants versus the average mean of the IRFP applicants. 

The number of international postdoctoral fellowships is a measure of internationalization and not a 
measure of employment. 

This figure includes the IRFP postdoctoral fellowship. 
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Exhibit 5.1: Fellows' and Unfunded Applicants' Engagement in International Collaboration 



Awardee 

Unfunded 


Impact 



Adjusted 

Adjusted 

Impact 

Standard 


Outcomes 

Mean 

Mean 

Estimate 

Error 

P value 

Work with individuals outside the U.S. 


Number of international postdoctoral fellowships a 

1.10 

0.63 

q 47*** 

0.08 

0.000 

Any employment outside the U.S. between 2 years 
after IRFP application and October 1, 2010 b 

0.45 

0.56 

-0.11 

0.07 

0.131 

Duration of any employment outside the U.S. during 
the reference period h 

4.10 

3.14 

0.96 

0.75 

0.201 

In current job (as of October 1, 2010), works with 
individuals located in other countries c 

0.64 

0.57 

0.07 

0.06 

0.200 

In current job, work with individuals in other 
countries includes joint publications and/or jointly- 
developed products 1 

0.54 

0.50 

0.04 

0.06 

0.461 

Publications with foreign collaborator 

Number of publications co-authored with a foreign 
collaborator 11 

12.76 

6.69 

6.07** 

1.99 

0.002 

Percent of publications co-authored with a foreign 
collaborator 6 

37.17 

26.50 

10.67** 

3.69 

0.004 

Fostering international collaboration 

Has mentored others from the U.S. traveling to 
another country to conduct research f 

0.59 

0.58 

0.01 

0.07 

0.886 

Engages in one or more activities to foster 
international collaboration 6 

0.72 

0.64 

0.09 

0.07 

0.204 

*p<.05, **p<.01, ***p<.001 






EXHIBIT READS: IRFP Awardees (who had completed their fellowship by October 1, 2010) held, on average, 1.10 
international postdoctoral fellowships, compared to Unfunded Fellows, who held, on average, .63 international 


postdoctoral fellowships. 






NOTES: Only IRFP Awardees who had completed their IRFP fellowship by October 1, 2010 (including any U.S. -based "re- 
entry period") were included in these analyses. Notes below show other restrictions for individual outcomes listed. 
a The IRFP award was included in the count of international postdoctoral appointments. N=937 (387 Awardees, 550 


Unfunded Applicants); Missing=10 (3 Awardees, 7 Unfunded Applicants). 

b N=824 (367 Awardees, 457 Unfunded Applicants); Missing=33 (6 Awardees 27, Unfunded Applicants). For this analysis, 
"working at the proposed foreign host institution at time of application" (Item C5a) was included as a control. Only IRFP 
awardees who had applied prior to 2008 were included. 
h N=375 (167 Awardees 208, Unfunded Applicants); Missing=0. 

c N=930 (370 Awardees 560, Unfunded Applicants); Missing=10 (3 Awardees, 7 Unfunded Applicants). 

1 N=937 (387 Awardees 550, Unfunded Applicants); Missing=22 (12 Awardees, 10 Unfunded Applicants). 
d Publications included peer-reviewed journal articles, peer-reviewed conference publications (e.g. abstracts, conference 
papers, posters), and book chapter(s) (e.g., in edited volumes). N=937 (387 Awardees, 550 Unfunded Applicants); 
Missing=44 (13 Awardees, 31 Unfunded Applicants). 

e Publications included peer-reviewed journal articles, peer-reviewed conference publications (e.g. abstracts, conference 
papers, posters), and book chapter(s) (e.g., in edited volumes). N=937 (387 Awardees, 550 Unfunded Applicants); 
Missing=48 (12 Awardees, 36 Unfunded Applicants). 

f Only IRFP awardees who had applied prior to 2008 were included. N=824 (367 Awardees, 457 Unfunded Applicants); 
Missing=25 (14 Awardees, 11 Unfunded Applicants). 

e These activities included one or more of the following: Established a program to foster international collaborations; 
hosted researchers or professional colleagues from another country at my institution; led a delegation of colleagues to 
visit a research laboratory, university, or business in another country; established or served in a leadership role for an 
international association for professionals in my line of work. Only IRFP awardees who had applied prior to 2008 were 
included. N=824 (367 Awardees 457, Unfunded Applicants); Missing=25 (14 Awardees, 11 Unfunded Applicants). 
SOURCE: IRFP Applicant Survey-Items C9, Dla, Dlb, D4, D4a, D6, D7, D7a, D8, D10. 
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5.2.2 Other Career Outcomes 

The evaluation explored whether the IRFP experience in a foreign country represented a detour with 
potentially negative consequences for fellows' career paths, and whether or not IRFP affected the 
type of employment IRFP fellows pursue and their professional outcomes (with a particular 
emphasis on fellows employed in academia). Although the IRFP award is not expected to affect 
these outcomes, it is important to test whether or not receiving an IRFP fellow has unintended 
adverse effects on professional outcomes. 

There were no statistically significant differences between IRFP fellows and unfunded applicants on 
career outcomes. These findings are important for the IRFP program given criticism that time spent 
abroad may put individuals at a disadvantage relative to their peers who do not engage in these 
activities. 59 Testing the impact of IRFP on other career outcomes was motivated primarily to 
determine whether any observed benefits of IRFP resulted in trade-offs in terms of other 
professional accomplishments. The IRFP goal of fostering international collaboration is not intended 
to come at the expense of scholarly productivity, professional opportunities, or career 
advancement. 

The findings on career outcomes demonstrate that IRFP does not negatively impact the careers of 
fellows who engage in a research sojourn abroad. That is, the IRFP experience does not detract from 
subsequent career productivity and progression along career paths. Indeed, the average number of 
adjusted publications for fellows (34 publications) was higher than the average for unfunded 
applicants (27 publications), although this difference was not statistically significant (no exhibit). 

There were no statistically significant differences in the types of employers of former IRFP fellows or 
unfunded applicants. Fellows were no more or less likely than unfunded applicants to work at an 
educational institution; to be self-employed or a business owner; or to work in the private sector or 
in public service either in local or state government, in the U.S. Federal government, or in a branch 
of the U.S. military services. These findings suggest that receiving an IRFP award did not have a 
measurable effect on the types of employment an individual applicant would have pursued without 
having completed an international postdoctoral fellowship. 

Similar proportions of fellows and unfunded applicants (62 percent each) held a position as a 
research faculty member, scientist, associate or fellow during the reference week. 60 It is important 
to note that individuals who held a research position may also have simultaneously held other types 
of academic positions, including that of a teaching faculty member, a department chairperson, or 
academic dean. 61 Finally, there were no statistically significant differences with respect to faculty 
rank or tenure status between IRFP fellows and unfunded applicants. 


NSB. 2001. 

60 The reference week was the week of October 1, 2010. 

61 To ensure that IRFP Fellows and unfunded applicants had an equivalent amount of time to achieve 
professional and career milestones, all of the analyses reported here controlled for the amount of time 
that had elapsed since earning the doctorate. 
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Exhibit 5.2: Types of Employers of Fellows and Unfunded Applicants 


Employed in 

Awardee 

Adjusted 

Mean 

Unfunded 

Applicants 

Adjusted 

Mean 

Impact 

Estimate 

Impact 

Standard 

Error 

p value 

Educational institution 

0.76 

0.71 

0.06 

0.06 

0.327 

Self employed 

0.02 

0.01 

0.02 

0.02 

0.290 

Private sector 

0.11 

0.16 

-0.04 

0.04 

0.321 

State or local government 

0.01 

0.01 

0.00 

0.01 

0.671 

Federal government 

0.06 

0.06 

-0.01 

0.03 

0.854 

Other 

0.03 

0.06 

-0.03 

0.02 

0.203 


*p<.05, **p<.01, ***p<.001 

EXHIBIT READS: 76.1 percent of IRFP awardees (who had completed their IRFP fellowship by October 1, 2010) were 
employed at an educational institution during the week of October 1, 2010 compared to 70.5 percent of unfunded 
applicants. This difference of 5.6 percentage points was not statistically significant. 

NOTES: These items were answered by awardees (who had completed their IRFP fellowship) and unfunded applicants who 
were working during the week of October 1, 2010 for pay or profit in a non-postdoctoral appointment. N=722 (322 
Awardees, 400 Unfunded Applicants), Missing=0. 

SOURCE: IRFP Applicant Survey-Items Die, D2 and D3. 
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Exhibit 5.3: Fellows' and Unfunded Applicants' Career Outcomes 


Outcome 

Awardee 

Adjusted 

Mean 

Unfunded 

Adjusted 

Mean 

Impact 

Estimate 

Impact 

Standard 

Error 

P value 

Held two or more total postdoctoral 
appointments 3 

0.60 

0.51 

0.09 

0.06 

0.111 

Most prestigious grant, award, or honor for 
research was from an international 
organization or foreign government b 

0.22 

0.19 

0.03 

0.06 

0.603 

Number of post-application publications c 

34.00 

27.40 

6.60 

3.78 

0.081 

Currently holds a research faculty, scientist, 
associate or fellow position (solely or along 
with another type of academic position, e.g., 
teaching faculty, department chair, etc.) d 

0.62 

0.62 

-0.00 

0.07 

0.422 


Currently has a faculty rank of Assistant, 
Associate or Full Professor 6 

0.85 

0.90 

-0.06 

0.05 

0.424 

Currently has tenure ' 

0.55 

0.49 

0.06 

0.08 

0.444 


*p<.05, **p<.01, ***p<.001 

EXHIBIT READS: 59.7 percent of IRFP awardees (who had completed their IRFP fellowship by October 1, 2010) had held two 
or more postdoctoral appointments compared to 50.8 percent of unfunded IRFP applicants. This difference of 8.9 percent 
was not statistically significant. 

NOTES: Impact models included only those IRFP fellows who had completed their IRPF fellowship as of October 1, 2010. 
Other exclusions are noted below. 

3 N=937 (387 Awardees, 550 Unfunded Applicants), Missing=0. 

b N=657 (367 Awardees 290 Unfunded Applicants), Missing=28 (8 Awardees, 20 Unfunded Applicants). Includes applicants 
who had received grant(s) (as a principal investigator or co-principal investigator), prestigious awards or honors based on 
their research. 

c Publications included: peer-reviewed journal articles, peer-reviewed conference publications (e.g. abstracts, conference 
papers, posters), and book chapter(s) (e.g., in edited volumes). N=894 (375 Awardees, 519 Unfunded Applicants), 

Missing=43 (12 Awardees, 31 Unfunded Applicants). 

d Respondents could hold other concurrent positions at their academic institution (e.g. president, provost, chancellor, dean, 
department head or chair, teaching faculty, adjunct faculty). N=535 (247 Awardees, 288 Unfunded Applicants), Missing=0 . 
Includes applicants who were employed during the week of October 1, 2010 at an educational institution (other than a 
preschool, elementary, middle, or secondary school or system) during the week of October 1, 2010. 

c N=495 (232 Awardees, 263 Unfunded Applicants), Missing=l Unfunded Applicant. Includes applicants who were employed 
during the week of October 1, 2010 at an educational institution with, and in a position with, a faculty rank system. Excludes 
applicants working at a preschool, elementary, middle, or secondary school or system. 

f N=464 (217 Awardees 247 Unfunded Applicants), Missing=l Unfunded Applicant. Includes applicants who were employed 
during the week of October 1, 2010 at an educational institution with, and in a position with, a tenure system. Excludes 
applicants working at a preschool, elementary, middle, or secondary school or system. 

SOURCE: IRFP Applicant Survey-Items C9 and Dla, Die, D2, D2a, D2c, D2d, and D6. 


5.3 Comparisons of IRFP Fellows to a National Sample of Doctoral 
Recipients 


To situate IRFP fellows within a broader context, comparisons were made between fellows and 
respondents to the Survey of Doctoral Recipients (SDR). The SDR is a longitudinal survey of a 
nationally representative sample of science, engineering, and health (SEM) doctorate recipients. 
Details of the sampling frame, the SDR data (2006 and 2008 waves), and the analyses conducted are 
provided in Appendix D. Findings from the comparison of all IRFP applicants to the SDR are 
presented in Appendix F, and reveal that IRFP applicants are unique in that they are engaged in 
more international collaborations, and they are successfully engaged in careers. 
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Given the overall purpose of the SDR (to describe general characteristics of doctoral education and 
early post-graduation employment for doctoral recipients), it was not designed specifically to 
address questions that might be important for the IRFP evaluation, including, for example, whether 
U.S. doctorate recipients in STEM fields co-author research publications with foreign collaborators. 
The only SDR question related to that type of outcome asked SDR respondents whether they had 
worked with individuals in countries outside the U.S. Other survey questions addressed more 
general career outcomes, specifically broad measures of employment, research productivity, and 
faculty rank and tenure for those working at institutions of higher education (IHEs). 

Because any comparisons between survey data from the SDR and from IRFP fellows are descriptive 
in nature, they do not address impacts of the IRFP. Thus, findings should be interpreted with an 
understanding that there may be uncontrolled initial differences between IRFP fellows and 
respondents to the SDR. The analyses reported in this chapter can take into account (that is, 
statistically control for) the amount of time that has elapsed between when respondents earned 
their degrees and completed a survey, but the analyses cannot account for differences in prevailing 
conditions at the time data were collected from IRFP fellows and SDR respondents; specifically, 
there may well be factors related to economic conditions and employment that differ between the 
years during which the SDR respondents completed surveys (2006 and 2008), and when IRFP fellows 
completed surveys in 2011. Also, the SDR excluded individuals living outside the U.S. for sustained 
periods, whereas the IRFP did not. Finally, because it was not possible to identify individual IRFP 
fellows in the SDR sample there could be overlap between the samples that renders the groups non- 
independent, but this overlap would be relatively small. 

Comparisons between SDR respondents and IRFP fellows were limited to SDR respondents who had 
completed a doctoral degree by the reference date specific to that SDR wave (April 1, 2006 in SDR 
2006; October 1, 2008 in SDR 2008); and to IRFP fellows who had applied for IRFP prior to 2008 
(N=379) and who had completed their IRFP fellowship by October 1, 2010. In addition, analyses 
controlled for gender, whether or not an individual was a member of an ethnic or minority group 
traditionally underrepresented in STEM fields (URM), field of study for the first doctorate, and the 
number of years between receipt of first doctorate and the reporting year of outcomes. Because the 
analytic samples are different, estimates for IRFP fellows below are different from the IRFP impact 
estimates presented above; for more detail please refer to Appendix D. 

5.3.1 Findings: IRFP Fellows vs. SDR Respondents 

The comparison of estimates on career outcomes of IRFP fellows to the national estimates 62 indicate 
that IRFP fellows' outcomes compare well against national STEM PhD holders on employment, 
publications, and international collaborations, suggesting that IRFP attracts a talented pool of 
applicants. As discussed above, these differences cannot be attributed to the IRFP program, but they 
provide a useful reference point. 


The estimates for former IRFP Fellows shown in comparisons to SDR respondents differ from the impact 
estimates comparing IRFP Fellows to unfunded applicants in the previous section. Covariates in the 
present analyses were grand mean centered using the average mean of the SDR and former IRFP Fellows 
versus the average mean of the IRFP Fellows. 
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Number of postdoctoral positions 

Former IRFP fellows had held, on average, a greater number of postdoctoral fellowships (1.7) than 
U.S. S&E doctoral recipients (.51), a statistically significant difference of 1.2. 63 This finding is not 
surprising: having received an IRFP fellowship, these individuals already held at least one 
postdoctoral appointment, and not all STEM doctorates pursue any postdoctoral appointments in 
their career. 

Employment 

Exhibit 5.4 illustrates the differences in employment circumstances between former IRFP fellows 
and the SDR sample. IRFP fellows were more likely to work in their current job with individuals in 
countries other than the U.S. (71 percent) than the typical STEM doctoral recipient (30 percent), a 
large and statistically significant difference. Virtually all former fellows and virtually all STEM 
doctoral recipients in the U.S. were employed (97 percent and 92 percent respectively), but IRFP 
fellows were more likely than the typical STEM doctorate to be working during a specified reference 
week, a statistically significant difference of 4.3 percentage points. 64 

Former IRFP fellows and SDR respondents also differed with respect to their types of employers. 
Fellows were more likely than the national pool of PhD holders to be employed at an educational 
institution (74 percent of fellows versus 46 percent of the SDR sample). Fellows were less likely than 
SDR respondents to be employed in the private sector (13 percent of fellows versus 34 percent of 
the SDR sample) or to be self-employed or a business owner (2 percent versus 9 percent). 


For comparisons of the number of postdocs held since receipt of first doctoral degree, the SDR 2006 wave 
of data were used as these data were not available in the 2008 wave. 

64 For IRFP Fellows, the reference week was the week of October 1, 2010; for SDR respondents, the week of 
October 1, 2008. 
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Exhibit 5.4: Characteristics of Employment of IRFP Fellows and SDR Respondents 


0 20 40 60 80 100 



Self-employed or business owner 


Federal government or military 
service 


Local or state government 


In current job, works with individuals 
in countries other than U.S. 




71.4*** 


*p<.05, * **p<.01, ***p<.001 

EXHIBIT READS: 96.7 percent of former IRFP fellows were currently employed during the reference week (October 1, 
2010) compared to 92.3 percent of SDR respondents (whose reference week was October 1, 2008). This difference of 4.3 
percentage points was statistically significant (p<.0001). 

NOTES: 


Currently employed: This item was answered by former IRFP fellows who had completed their IRFP fellowship (including 
any U.S.-based "re-entry period") as of October 1, 2010 (N=379, Missing=0) and by SDR 2008 respondents who had 
completed a PhD by October 1, 2008 (N=29,974, Missing=0). Private Sector, Self-Employed, Federal Government or 
Military Service, Local/State Government, Other Employer: This item was answered by all former IRFP fellows who had 
completed their IRFP fellowship (including any U.S.-based "re-entry period”) as of October 1, 2010 and were employed as 
of October 1, 2010 in a non-postdoctoral position (N=312, Missing=0) and by SDR 2008 respondents who had completed 
a PhD by October 1, 2008 and were employed during the week of October 1, 2008 (N=26,134, Missing=0). Items from 
which these data derive differed slightly between the IRFP Applicant Survey and the SDR 2008; thus, Local Government 
(city, county, school district) and State Government (including state colleges/universities) were combined into a single 
category for both groups; and U.S. Federal Government and U.S. Military service, activity duty or Commissioned Corps 
(e.g., USPHS, NOAA) were combined for both groups. 

In current job, works with individuals in other countries: This item was answered by former IRFP fellows who had 
completed their IRFP fellowship (including any U.S.-based "re-entry period”) as of October 1, 2010 and were employed 
during the week of October 1, 2010 (N=363, Missing=0) and by SDR 2006 respondents who had completed a PhD by April 
1, 2006 and were employed during the week of April 1, 2006 (N=27,119, Missing=0). This item was not included in the 
SDR 2008 wave. 

SOURCES: IRFP Applicant Survey-Items C9, Die, D3, and D4. SDR 2008-Items Al, All, and A12 and SDR 2006-Item A27 . 
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Among respondents working in an institution of higher education, an equal percent of fellows and 
SDR recipients were in a tenure track positions (e.g., Assistant, Associate, or Full Professor) (93 
percent each, Exhibit 5.5). However, a greater percentage of former fellows working at these 
institutions had earned tenure (78 percent versus 56 percent); this difference was statistically 
significant. 


Exhibit 5.5: Characteristics of Employment in Educational Institutions for Fellows versus SDR 
Respondents 


0 20 40 60 80 100 


Assistant, Associate, Full Professor 


Tenured 



■ IRFP Fellows 

■ SDR Sample 


*p<.05, * **p<.01, ***p<.001 

NOTES: These items were answered by former IRFP fellows who had completed their IRFP fellowship (including any U.S.- 
based "re-entry period") as of October 1, 2010, who were working in an educational institution during the reference 
week (Oct 1, 2010) and who did not report working in a preschool, elementary, or secondary school system, and by SDR 
respondents who had completed a PhD by October 1, 2008, were working in an educational institution during the week 
of October 1, 2008, and who did not report working in a preschool, elementary, or secondary school system (Item A13). 
Fellows and SDR respondents indicating that their institution or their position did not have a faculty rank were excluded. 
(IRFP: N=232, Missing=15; SDR 2008: N=9,501, Missing=0); fellows and SDR respondents indicating that their institution 
or their position did not have a tenure system were excluded (IRFP: N=215, Missing=16; SDR 2008: N=8,811, Missing=0). 
SOURCE- IRFP Applicant Survey-Items C9, Die, D2, D2a, D2c, and D2d; SDR 2008-Items Al, A12, A13, A15, and A16.. 


Among those individuals employed at an IHE, Exhibit 5.6 below shows the percentages of former 
fellows and SDR respondents who held one or more of several different types of academic roles. 
Individual employees reported that they hold multiple roles with different responsibilities, reflecting 
common roles/responsibilities in IHEs. A statistically significant higher percentage of former IRFP 
fellows than SDR respondents: held an academic position as a research faculty member, scientist, 
associate or fellow; held an academic position as a teaching faculty member; and held a type of 
academic position other than those listed. Other differences were not statistically significant. 


Abt Associates Inc. 


5. Outcomes of Participation in IRFP 72 


Evaluation of International Research Fellowship Program: Final Report 


Exhibit 5.6: Types of Academic Positions Held (Solely or Along with Another Position) by IRFP 
Fellows versus SDR 


Research faculty, scientist, 
associate, fellow (solely or 
along with another position) 

Teaching faculty (solely or 
along with another position) 

Adjunct faculty (solely or 
along with another position) 

Other academic position (solely or 
along with another position) 

Dean, Department Head/Chair (solely or 
along with another position) 


*p<.05, **p<.01, ***p<.001 

EXHIBIT READS: Among those working at an educational institution, 68 percent of former IRFP fellows versus 43 percent 
of the SDR 2008 respondents held a research position (solely or concurrently with another position); this difference was 
statistically significant. 

NOTES: The academic positions are not mutually exclusive: Individuals could select more than one response. For 
example, individuals could hold a research faculty position and a teaching faculty position simultaneously. 

These items were answered by former IRFP fellows who had completed their IRFP fellowship (including any U.S.-based 
"re-entry period") as of October 1, 2010, were working at an educational institution (other than in a postdoctoral 
position) during the week of October 1, 2010 and who did not report working in a preschool, elementary, middle, or 
secondary school or system (N=247, Missing=0) and by SDR respondents who had completed a PhD by October 1, 2008, 
who were working in an educational institution during the week of October 1, 2008, and who did not report working in a 
preschool, elementary, or secondary school system (N=ll,773, Missing=0). 

SOURCES: IRFP Applicant Survey-Items C9, Die, D2, D2a and D2b. SDR Survey 2008-Items Al, A12 and A14. 

IRFP fellows also reported having statistically significantly more publications than SDR respondents. 
For example, IRFP fellows produced 11 peer-reviewed journal articles on average, compared to an 
average of 5 produced by SDR respondents (Exhibit 5. 7). 65 However, SDR respondents had more 
patents compared to IRFP fellows (0.8 versus 0.4 patents). 



Statistical models controlled for the different time periods about which respondents reported 
publications. 
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Exhibit 5.7: Mean Number of Publications for Former IRFP Fellows vs. SDR Respondents 3 


0 2 4 6 8 10 12 


Peer-reviewed journal articles (IRFP) 
Articles in refereed professional journal (SDR) 


Peer-reviewed conference publications 
(abstracts, papers, posters) (IRFP) 
Papers for regional, national or 
international conferences (SDR) 



Book chapter(s) (e.g., in edited volumes) (IRFP) 
Books or monographs (SDR) 



■ IRFP Fellows 


Patents, registered or pending (IRFP) 

US Patents granted to you as inventor and 
applications for US patents naming you as inventor 
(SDR) 



■ SDR sample 


*p<.05, **p<.01, ***p<.001 

EXHIBIT READS: The IFRP fellow's model adjusted mean for the outcome "Number of articles published" was 23.32. The 
model-adjusted mean for the SDR sample was 4.91. IRFP fellows had 18.4 more articles published than the SDR sample. 
This difference was significantly different than zero (p=<0.0001). 

NOTES: The IRFP benchmarking estimate is different from the IRFP impact estimate because in the benchmarking 
analysis, covariates are grand mean centered using the average mean of the SDR and IRFP fellows versus the average 
mean of the IRFP fellows. This item was answered by Awardees who as of October 1, 2010 had completed IRFP (including 
any U.S.-based "re-entry period"; N=381, Missing=7 to 9 Awarded Applicants). SDR: This item was answered by SDR 
survey takers who had completed a PhD by October 1, 2008 (N=29,953, Missing=0). 

a SDR asks about publication/patents since 2003 (only in the last 5 years) while IRFP asks about all publication/patents 
since application to IRFP. Therefore, we should interpret these finding with caution. 

Source: IRFP Applicant Survey-Items C9 and D6. SDR Survey 2008-Item Cl. 


Fellows’ and Hosts’ Perception of the Outcomes of IRFP 


This last section contributes to a deeper understanding of the outcomes and benefits of the IRFP 
program. It supplements the findings in the preceding sections by exploring former IRFP 
participants' reports of their post-fellowship experiences and the outcomes of the program. Below, 
findings based on former IRFP fellows' survey responses describe the transitions from IRFP to the 
next stage of their respective careers and the role of IRFP in those transitions. Fellows and hosts 
describe the publications and professional benefits they attribute to IRFP, whether or not they have 
sustained their collaboration since the end of the postdoctoral period, and reasons why their 
collaboration has persisted or not; and both groups discuss how the benefits they derived from IRFP 
have extended to others. 


Abt Associates Inc. 


5. Outcomes of Participation in IRFP 74 




Evaluation of International Research Fellowship Program: Final Report 


5.4.1 Entering the Workforce 

IRFP fellows had to secure employment upon reentry to the U.S. Virtually all former IRFP fellows 
(i.e., those who had completed their fellowship by October 1, 2010) had found employment (97 
percent) and a large majority (68 percent) were working in a 
university setting. 66 


expected. Less than 3 percent of former fellows felt that they 

had lost an important career or educational opportunity by participating in IRFP. Less than 1 percent 
of IRFP fellows indicated that participating in IRFP constrained their options. 67 


University setting included a 4-year college or university, medical school (including university-affiliated 
hospital or medical center), or a university affiliated research institute (IRFP Applicant Survey, Item D2a). 

No exhibit. Source: IRFP Applicant Survey, Item F2a 



The IRFP was a major contributing 
factor to obtaining my tenure-track 
faculty position. It also was a 


Most former IRFP fellows reported that they reaped positive 
career outcomes from their participation in IRFP. About one- 
third of fellows (32 percent) indicated that IRFP had caused 
them to consider professional opportunities they would not 
have otherwise (Exhibit 5.8). Thirty percent of former IRFP 


defining period in development of 
my research and the experience has 
provided me with a lot of the tools 
and skills I need to run a successful 
— search program. (IRFP fellow) 


fellows reported that obtaining full-time employment after the 
conclusion of their IRFP fellowship was more difficult than 
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Exhibit 5.8: Fellows' Perceptions of How IRFP Affected Their Post-Fellowship Career Options 


Fellows' Report of IRFP's Effects on Broadening Their Range of Post-Fellowship 
Opportunities a 

Percent of Fellows 
(n=378) 

IRFP broadened my career options 

78.8 

Not sure 

13.7 

IRFP did not broaden my career options 

7.4 

Fellows' Perceived Effects of IRFP on Career Opportunities & Goals 

Percent of 
Fellows b,c,d 

IRFP made me more competitive for jobs 1 was interested in 0 

67.7 

IRFP made me consider professional opportunities 1 would not have considered in 
the past d 

31.9 

Obtaining full-time employment after the conclusion of my IRFP fellowship was more 
difficult than 1 expected b 

29.6 

1 decided to pursue research in a different discipline than the one 1 was most familiar 
with when 1 began my IRFP fellowship d 

14.0 

My career goals changed from an academic to a non-academic career I 11 

8.1 

My career goals changed from a non-academic to an academic career 11 

4.1 

1 lost an important career or educational opportunity by participating in IRFP c 

2.9 


NOTES: Only IRFP fellows who had completed their fellowship by October 1, 2010 answered these items. 
a Missing=6. 

b N=373, Missing=ll Fellows. 
c N=367 Missing=17 Fellows. 
d N=378 Missing=6 Fellows. 

SOURCE: IRFP Applicant Survey-Items F2, F3, F6. 

A large number (454) of respondents provided additional comments about how IRFP had broadened 
their career options, and the excerpts presented below illustrate the range of such comments. 

IRFP allowed me to meet researchers from around the world, broadened my experience working 
in different systems using different methods, expanded my theoretical background, and 
sharpened my critical thinking skills by forcing me to guestion how and why I conduct my 
research. (IRFP fellow) 

I was exposed to a niche of physics where I had the chance to become a pioneer and an expert. I 
have applied this knowledge to other fields, in particular mathematical finance, where I have 
been able to use those tools from physics to solve many useful problems. (IRFP fellow) 

I had an opportunity to develop in my field of research and explore options for continued work in 
[country]. These options were not as visible at that time (internationally) and allowed me to 

remain associated with natural history collections-based research. There have been interesting 
developments in biodiversity informatics in [country] during the past few years and I have had 
the privilege to participate in several initiatives and projects. I found opportunities to apply 
results from these developments to my activities, with IRFP (and post-1 RFP) skills and experience 
noted as highly desired in positions advertised by American and international research 
institutions. (IRFP fellow) 
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It gave me a broader understanding of research in my field, and exposed me to some top-notch 
methods that I then applied to my own research, with some highly successful results. It also 
provided me a way to meet many influential scientists in Europe. (IRFP fellow) 


5.4.2 Professional and Other Benefits 


Former fellows also cited other professional benefits of 
participation in IRFP. More than two-thirds (71 percent) made 
important connections to researchers in their host country. A 
majority of fellows also credited their IRFP fellowship with 
opening up new areas for investigation (71 percent), with 
allowing them to make substantial advancements in their 
research (64 percent), and with making them familiar with the 

scientific enterprise in their host site (65 percent). Half of all former fellows said that participation in 
IRFP made them committed to international research collaboration (Exhibit 5.9). Only 2 percent said 
that they were more skeptical about international collaboration than before participation in IRFP. 68 


It provided me with an example of 
how to lead a group, interact most 
effectively with collaborators and keep 
everyone involved on a project 
interested in the work. (IRFP fellow) 


Most fellows also reported a range of personal benefits, including making personal connections in 
their host country (93 percent), becoming comfortable with another country's culture and 
traditions, (84 percent), and gaining proficiency in a foreign language (42 percent). Moreover, nearly 
one-third (33 percent) decided to live outside the U.S. for six months or more after the conclusion of 
their fellowship. 69 


Exhibit 5.9: Fellows’ Reported Professional Benefits of the IRFP Fellowship 



Percent of 
Fellows (N=378) 

1 made valuable connections to researchers in the host country 

70.6 

My work at the host institution opened up new areas of investigation 

70.5 

1 became familiar with scientific enterprise of the host country 

65.0 

My work at the host institution resulted in a substantial advancement in my research 

63.9 

1 became committed to international research collaboration 

49.9 

None of the above 

3.8 


NOTES: Missing Data: 6 Awarded Applicants. Responses do not sum to 100 percent because multiple choices were 
permitted. 

SOURCE: IRFP Applicant Survey-Item F3. 


Former hosts also cited a number of positive outcomes of their collaboration with the IRFP fellow. 
The most frequently cited benefit hosts derived was publishing papers and presenting work at 
professional conferences: 65 percent of hosts reported that they (or other members of their 
research group) published papers based on their collaboration with the IRFP fellow, and 56 percent 
said that one or more conference presentations came about based on this work. In addition, 47 
percent of hosts credited the collaboration with acquiring new methodological techniques or new 


No exhibit. Source: IRFP Applicant Survey-Item F6. Only IRFP Fellows who had completed their fellowship 
by October 1, 2010 answered this item (N=384; Missing Data=17). 

No exhibit. Source: IRFP Applicant Survey-Item F5. Only IRFP Fellows who had completed their fellowship 
by October 1, 2010 provided data on personal benefits (N=378; Missing Data=6) 
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theoretical approaches. For nearly one-third of hosts, working with an IRFP postdoctoral fellow 
enhanced their interest in collaborating with researchers from the U.S. Finally, for 12 percent, 
hosting an IRFP fellow was their first ever collaboration with a U.S. -trained researcher. 


Exhibit 5.10: Hosts' Reported Benefits of Collaborating with IRFP Fellow 


Hosts' Perceived Benefits of Working with IRFP Fellow 

Percent of Hosts 
(N=326) 

Published papers based on work with this fellow 

65.0 

Gave one or more conference presentations based on the collaborative work 

55.8 

Learned new methodological/analytical techniques or theoretical approach 

47.2 

Enhanced my interest in collaborating with U.S. researchers 

32.8 

Changed the direction of some research projects in my group 

31.9 

Established or renewed collaboration(s) with other U.S. researchers 

31.3 

Obtained funding based on the collaborative work 

26.1 

Other U.S. researchers were more interested in collaborating with me 

22.4 

Became more familiar with the research enterprise in the U.S. 

20.6 

Hosting an IRFP fellow was the first time 1 had collaborated with a researcher 
trained in the U.S. 

12.3 

None 

8.9 

Improved my ability to communicate in English 

8.6 


NOTES: Missing Data: 2 Hosts. Responses do not sum to 100 percent because multiple choices were permitted. 
SOURCE: IRFP Host Survey-Item E2. 


Hosts' elaborations provide some insight into these benefits. 

Collaboration was great and we have identified a very interesting area of collaboration. 
Interaction with the fellow in small meetings and in lab meetings was great. She has now a very 
strong and interesting database with which we will start publishing and will develop new 
collaborations. It was interesting to see the different approaches to many logistic aspects of 
research and to learn from these differences. The Postdoctoral fellow stimulated my students to 
think in different directions, to try different statistical analyses. She helped them with their 
English writing. (IRFP Host) 

[The fellow] contributed to the intellectual life in my laboratory by interacting with my graduate 
students, demonstrating laboratory technigues to them, and providing a broader perspective on 
the research field. The work we did was successful although it still has not reached the 
publication stage. (IRFP Host) 

There was an excellent confluence between the group's direction and the background of the 
fellow, which resulted in a scientifically productive project which benefited all. (IRFP Host) 
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5.4.3 Sustained Collaborations 


Of the former fellows (1992-2009) who had completed their 
IRFP postdoctoral fellowship, 46 percent had since 
collaborated on research with their former host, and an 
additional 46 percent had communicated with their host after 
the fellowship period. 70 During the continued collaborations 
(Exhibit 5.11), former fellows and hosts co-authored papers 
(82 percent), exchanged ideas, data, results or tools (80 
percent) and visited each other at their respective institutions 
(44 percent). In some cases, continued collaboration extended 
to co-advising students (25 percent). Eleven percent of former 
fellows reported that they held a position with their former 
host's group, and 9 percent held a position at the same institution 


My career was completely changed 
with that experience. I believe that 
research experience was the major 
factor in every career move I have 
made since then. Currently I have 
numerous collaborators from that 
research group, as well, and this 
continues to enrich my life/ 
professional experience (IRFP fellow) 


as their former host. 


Exhibit 5.11: Types of Collaboration between Fellows' and Hosts after the End of the IRFP 
Fellowship 


We co-authored papers 

We exchanged ideas, data, ideas, 
research results, or tools 

We visited each other at our institutions 
We co-advised students 
Other 

I have a position in the host's group 
I have a position in the host's institution 



NOTES: Only IRFP fellows who had completed their fellowship by October 1, 2010 and who reported having collaborated 
with their former host on a research project since the end of their IRFP fellowship answered this item (N=179, 
Missing=0). Results do not sum to 100 percent because multiple choices were permitted. 

SOURCES: IRFP Applicant Survey-Item Fla. 


No exhibit. Source: IRFP Applicant Survey-Item FI. Former IRFP Fellows were asked to report any 
collaboration or communication with their host since the end of the fellowship and October 1, 2010, the 
reference date used throughout the IRFP Applicant Survey (N=378, Missing=6). 
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More than half of fellows still collaborating with their host reported that the most recent 
collaboration with their former host occurred within the past six months (52 percent, Exhibit 5.12). 


Exhibit 5.12: Timing of Most Recent Communications and Collaborations 


How Recently Did the Most Recent Collaboration Occur? 

Percent of Fellows 
(n=179) 

Within the past 6 months 

52.1 

3 or more years ago 

25.7 

1-2 years ago 

12.9 

Within the past 12 months 

9.4 


NOTES: Percents reported are weighted to reflect non-response. Only fellows who had completed their IRFP postdoctoral 
fellowship by October 1, 2010 and who reported having collaborated with their former host on a research project since 


the end of their IRFP fellowship were asked these questions. Missing Data: 6 Awarded Applicants. 

SOURCE: IRFP Applicant Survey-Items FI, Fib. 

Among those fellows reporting a collaboration within the past six months, 53 percent were from the 
1992 to 2004 IRFP cohorts (i.e., the first 12 years of the program); nearly half (47 percent) were 
from the 2005 to 2009 IRFP cohorts, the most recent four years of the IRFP program for which data 
were collected. More than half of those reporting that their most recent collaboration was three or 
more years ago were fellows from the 1992 to 1997 cohorts (Exhibit 5.13), and not surprisingly, no 
fellows from the 2004 to 2007 cohorts reported recent collaborations with former hosts three or 
more years ago. 71 


Note that IRFP Fellows from the 2008 and 2009 cohorts could not have had a collaboration with the most 
recent contact taking place three or more years prior to the survey reference date of October 1, 2010; any 
collaboration after the end of their fellowship would logically have had to occur more recently than three 
or more years ago. 
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Exhibit 5.13: Former IRFP Fellows' Most Recent Collaboration with Former Hosts by Cohort Year 

100.0 

80.0 

60.0 

40.0 

20.0 
0.0 

■ Within past 6 months ■ Within past 12 months ■ 1-2 years ago 3 or more years ago 



EXHIBIT READS: Among those IRFP fellows in the 1992 cohort who reporting collaborating with their former host since 
the end of their IRFP fellowship, 66 percent had done so within the past 1-2 years and 34 percent had done so 3 or more 
years ago. 

NOTES: Only fellows who had completed their IRFP postdoctoral fellowship by October 1, 2010 and who had 
collaborated with their former host since the end of IRFP fellowship were asked this question (N=179, Missing=6). 
SOURCE: IRFP Applicant Survey-Item Fib. 


The reasons fellows cited for not pursuing post-fellowship collaboration included diverging research 
interests (51 percent, Exhibit 5.14), being too busy (34 percent), a lack of action by the host to 
pursue a collaboration with the fellow (28 percent), and either the fellow or host lacked sufficient 
funding (25 percent). One-fifth of fellows reported that that further collaboration would not benefit 
them. 
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Exhibit 5.14: Reasons IRFP Fellows Have Not Collaborated with Former Host 


Reasons Indicated 

Percent of 
Fellows 

Our research interests diverged 

51.2 

One (or both) of us is too busy with other projects 

34.1 

My host did not actively pursue or maintain further collaboration with me 

27.8 

One or both of us lacked funding needed to maintain collaboration 

24.9 

1 did not think that further collaboration would be beneficial for me 

20.3 

Other 

19.7 

Geographic distance has hindered further collaboration 

18.2 

Political or cultural differences have hindered further collaboration 

1.4 

Language differences have hindered further collaboration 

0.0 


NOTES: Only fellows who had completed their IRFP postdoctoral fellowship by October 1, 2010 and indicated (Item FI) 
that they had not collaborated with their host were asked this question (N=199, Missing=0). 

SOURCE: IRFP Applicant Survey-Item Flc. 


5.4.4 Post-IRFP Publications 

IRFP fellows were productive researchers during and after their fellowship, producing an average of 
31 publications (including peer-reviewed journal articles, conference publications and book 
chapters). Moreover, 40 percent of these publications were produced with a foreign co-author— and 
26 percent were produced with their IRFP host (Exhibit 5.15). 
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Exhibit 5.15: IRFP Fellows Post-IRFP Publications 



Mean 

Standard 

deviation 

Mean number of post-IRFP publications 

Patents, registered or pending 

0.6 

2.8 

Peer-reviewed journal articles 

16.4 

20.7 

Peer-reviewed conference publications (e.g., abstracts, conference 

12.7 

26.7 

papers, posters) 



Book chapter(s) (e.g., in edited volumes) 

1.5 

3.6 

Total publications (excluding patents) 

30.5 

41.3 

Total publications with foreign colleague (excluding patents) 

12.5 

23.1 

Total publications with IRFP host (excluding patents) 

5.1 

8.5 

Mean percent of post-IRFP publications with foreign collaborator 

Patents, registered or pending 

2.1 

15.5 

Peer-reviewed journal articles 

39.3 

36.7 

Peer-reviewed conference publications (e.g., abstracts, conference 

28.2 

40.7 

papers, posters) 



Book chapter(s) (e.g., in edited volumes) 

14.1 

35.2 

Mean percent of all publications (excluding patents) 

40.0 

34.9 

Mean percent of post-IRFP publications with former IRFP host 

Patents, registered or pending 

0.7 

8.7 

Peer-reviewed journal articles 

25.2 

33.9 

Peer-reviewed conference publications (e.g., abstracts, conference 

18.6 

34.8 

papers, posters) 



Book chapter(s) (e.g., in edited volumes) 

7.3 

25.6 

Mean percent of all publications (excluding patents) 

26.4 

33.3 


EXHIBIT READS: Former IRFP fellows who had completed their fellowship by October 1, 2010 produced an average of 
0.6 patents between the time of their application and October 1, 2010. 

NOTES: Weighted means and standard deviations are reported. Unweighted N reported. Only IRFP fellows who had 
completed their fellowship by October 1, 2010 answered this item. 

SOURCE: IRFP Applicant Survey-Item D6. 


5.5 Effects of IRFP Extending Beyond the Fellow and Host 


This section explores how the benefits of IRFP 
diffused beyond the fellow to his or her colleagues, 
the effect of IRFP on former hosts' likelihood of 
subsequent collaboration with their former 
postdoctoral colleagues, and other benefits hosts 
reported via open-ended comment. Once IRFP 
fellows returned to the U.S., they have had the 
opportunity to share with colleagues any new skills, 
data, or methods acquired during their IRFP 
fellowship, and they have the opportunity to foster 
new international collaborations among their 
colleagues. 


I recommend the program most highly to 
every grad student who comes through my 
office (I have also led informal postdoc 
workshops where I strongly encourage 
students to apply). The opportunity to spend 
an extended amount of time dedicated to 
research in a different cultural setting (in my 
case, with the top specialists in the field) was 
an incredible experience. I moved on to the job 
market with renewed energy, new 
perspectives on my research, and a greater 
appreciation for international collaboration 
(and a stronger CV...). (IRFP fellow) 
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More than three-quarters of former IRFP fellows shared resources or tools acquired during their 
postdoc abroad and taught colleagues, students or peers methods learned during this time (Exhibit 
5.16). 


Exhibit 5.16: Sharing of IRFP Benefits by IRFP Fellows 



Percent who 


Engaged in 

Activities Undertaken by Former IRFP Fellows' to Share Benefits of Their Fellowship 

Activity 

Taught colleagues, students, or peers research methods that you learned during your 
IRFP fellowship 

78.1 

Shared with your colleagues resources or tools that you collected developed during 
your IRFP fellowship 

75.4 


NOTES: Only fellows who had completed their fellowship by October 1, 2010 were presented with this item (N=384, 
Missing Data: 6 to 9 Felows). Results do not sum to 100 percent because multiple choices were permitted. 

SOURCE: IRFP Applicant Survey-Item D9. 


More than half of former IRFP fellows reported that the methods or ideas that they had learned 
benefited others at their institution (64 percent). Forty-one percent reported that samples or tools 
from their fellowship benefited others in their institution. Smaller proportions of fellows reported 
that their enthusiasm may have spread to others: 23 percent reported that their peers became 
interested in international collaboration and 15 percent reported that members of their research 
group in the U.S. began an international collaboration (Exhibit 5.17). 


Exhibit 5.17: Benefits to Others as Reported by IRFP Fellows 


Benefits of IRFP for Colleagues of former IRFP Fellows 

Percent 

Research methods or ideas that 1 learned benefited others in my institution 

64.1 

Sample that 1 collected or tools that 1 developed benefited others in my institution 

40.8 

My peers became interested in international collaboration 

23.3 

Others in my research group (in the U.S.) began an international research 
collaboration 

15.4 

Researchers that 1 met during my fellowship joined my research group in the U.S. 

8.6 

Other 

7.1 

None of the above 

19.4 


NOTES: Only IRFP fellows who had completed their fellowship by October 1, 2010 answered this item (N=378, 
Missing=6). Results do not sum to 100 percent because multiple choices were permitted. 

SOURCE: IRFP Applicant Survey-Item F4. 


Hosts also served as a mechanism for extending the effects of the program to other scientists. 
Twenty-five percent of former IRFP hosts reported that a collaboration with a U.S. researcher 
resulted from their participation in the IRFP program (Exhibit 5.18) 
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Exhibit 5.18: Hosts Collaborating With a Former IRFP Fellow's Colleague or With Other U.S. 
Researcher(s) 


| Percent j 

With this individual’s former faculty advisor 

14.7 

With graduate students who work with this former IRFP fellow 

8.3 

With postdoctoral fellows who work with former IRFP fellow 

8.0 

With other researchers who work with this former IRFP fellow 

26.5 

None of the above 

55.9 

Currently collaborating with other U.S. researchers 

82.1 

A collaboration with a U.S. researcher resulted from hosting the IRFP fellow 3 

24.6 


NOTES: These items were answered only by hosts whose IRFP fellows' fellowships had concluded (N=313, Missing=4) 
a This item was answered by hosts whose IRFP fellows' fellowships had concluded and who reported a current 
collaboration with other U.S. researchers (i.e., not a direct colleague of their former IRFP fellow; N=257, Missing=l) 
SOURCE: IRFP Host Survey-Items B7, D3a, D3b, D3d. 

These benefits of the program as well as the other findings from the evaluation are tied to the 
specific goals of the program in the concluding chapter. 
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6 Conclusions 


Findings from this evaluation provide ample evidence that IRFP fulfills its promise of providing 
opportunities for collaborations among U.S. and foreign researchers in settings abroad. Unique 
among NSF postdoctoral programs, IRFP provides early career post-PhD scientists the opportunity to 
engage in overseas scientific research and collaborations with scientists outside the U.S. This study 
of IRFP examined the characteristics and motivations of IRFP participants, the opportunities for 
research and professional growth provided through IRFP, and the professional outcomes related to 
the program. 

The evaluation found evidence that the IRFP program is meeting its goals to: 

• Introduce early career scientists and engineers to opportunities for international research 
collaboration; 

• Build research capacity and global perspective of participants; and 

• Forge long-term relationships between U.S. and foreign S&E researchers. 

Below we summarize contributions the program has made in each of these areas, and then discuss 
some general conclusions from the evaluation. 


6.1 Opportunities for International Research Collaboration 


IRFP is a selective program that offers international research opportunities to individuals who are 
interested in international research collaborations. Both unfunded applicants and successful fellows 
reported being attracted to the program because it offered them an opportunity to conduct 
research with individuals in foreign settings and at institutions outside the U.S. Foreign hosts 
reported that the proposed research topics were engaging, as was the potential to create 
international collaborations in their research group. 

IRFP attracts a talented pool of applicants from among the pool of national STEM PhD holders, who 
reported interest in the contributions that opportunities abroad may make in their own research. 
After their experiences, half of all former fellows said that participation in IRFP made them more 
committed to international research collaboration. 

IRFP fellows indicated that they engage in research collaborations while abroad, and an 
overwhelming majority of fellows were satisfied with the quality of research conducted as part of 
IRFP and the timing of the fellowship with respect to their career goals. Flosts and fellows agreed 
they worked collaboratively on developing project ideas and hypotheses, interpreting results, and 
planning follow-up work. A majority of fellows also credited their IRFP fellowship with allowing them 
to make substantial advancements in their research. 

Further, IRFP offers fellows broad range of professional opportunities beyond the opportunity to 
conduct quality research at their host institution, including opportunities to network with colleagues 
from other than their host institutions, attend lectures in their field, attend conferences, and give 
talks at their host institution. 
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Hosts provided positive comments about their experiences with IRFP and the fellows specifically. A 
majority agreed that their IRFP fellow had sufficient knowledge and expertise for the research 
conducted and that the fellow integrated well with their research group. Further, hosts noted that 
IRFP fellows compared favorably to other postdoctoral fellows with whom hosts had worked. 


6.2 Research Capacity and Global Perspectives 


A unique feature of IRFP is its focus on providing international research experiences for postdoctoral 
fellows. Individuals who participate in IRFP develop their knowledge and research skills, broaden 
their perspectives, and contribute to the global understanding and research enterprise more 
broadly. A majority of fellows credited IRFP with opening up new areas for investigation and 
familiarizing them with the scientific enterprise in their host site. Three-quarters of fellows also 
observed that IRFP qualified them for a broader range of career options, and two-thirds felt it 
contributed to making them more competitive for jobs. 

The evaluation provides evidence that these experiences lead to greater levels of international 
research engagement among fellows. Fellows were consistently higher than unfunded applicants on 
several facets of international research, including the number of international postdoctoral 
fellowships, the number of publications with a foreign co-author, and the percentage of publications 
with a foreign co-author. The fellows' research productivity is consistently higher across all three 
measures. 

Importantly, this international focus did not come at the expense of research productivity or career 
advancement. Specifically, fellows and their peers were equally likely to hold multiple postdoctoral 
appointments, and were equally productive researchers, equally likely to hold a faculty rank of 
assistant, associate, or full professor, and equally likely to be tenured. 

Further, the program demonstrates potential to reach beyond the immediate participants. More 
than three-quarters of former IRFP fellows shared resources or tools acquired during their 
international postdoctoral fellowship and taught colleagues, students or peers methods learned 
during this time. 


6.3 Long-Term Relationships between U.S. and Foreign Researchers 


The relationships developed during IRFP seed subsequent professional collaborations and activities. 
Many IRFP fellows developed professional relationships that endured beyond the fellowship period, 
either through subsequent collaborations with their hosts, and/or additional communications. The 
continuing collaborations reported by a large proportion of former fellows and hosts included co- 
authored papers and exchanged ideas, data, results or tools. They also visited each other at their 
respective institutions, and co-advised students. Also, there is evidence that the collaborations may 
be seeding networks, as fellows interact with hosts' research groups, and as hosts establish 
collaborations with other U.S researchers as a result of their participation in the IRFP program. 
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Conclusion 


The evaluation demonstrated that individuals 
derive important benefits from the program, 
both on an individual and a collective basis. 
Former fellows and hosts strongly endorsed the 
IRFP program, and would recommend IRFP to 
their students and colleagues. IRFP offers an 
opportunity for interested researchers to 
develop their international research 
collaborations early in their careers. The 
experiences during IRFP seed relationships that 
often are sustained and that generate 
international research collaborations across 
geopolitical boundaries. As such, IRFP has a 
central role in NSF's efforts to respond to the 
NSB's charge that NSF play a leadership role in 
international S&E research and education 
activities, and specifically in promoting 
international S&E among early career scientists 
and engineers . 72 


This is a great program. It helped launch my 
career. I didn 't know the quality of people I 
would be working with, but really got very 
lucky. They took me under their wing, and I 
met many researchers at several different 
institutions over the course of the fellowship. 
The country benefits greatly from a program 
like this. During the 13 years since completing 
the fellowship. I've been working on various ... 
research projects as a principal investigator. 
The small amount of funds that the U.S. 
government invested in me during the IRFP I 
believe enabled my subsequent research 
contributions to these government agencies 
and goals.... Keep funding this program. It has 
a great deal of payback. (IRFP fellow) 


NSB. 2000. 
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NSF Highlights: 2010 

Investigating Variation in Natural Zebrafish Populations 

Highlight ID: 21515, Version: AC/GPA 



another U.S. researcher (Dr. Richard Mayden, right), a 
graduate student from Bangladesh (Shobnom Ferdous, 
left), and an ichthyologist from Dhaka University in 
Bangadesh (Dr. Ahmed Abu Tweb Ahmed, third from 
left) to sample natural populations of zebrafish 
throughout Bangladesh. 

Credit: Matiul Mahmud Munna 

Image Provided bv awhitelev@nrc.umass.edu 

NSFForm1515 



Zebrafish captured in the wild in a stream near 
Chittagong, Bangladesh. 

Credit: Andrew Whiteley, University of Massachusetts 
Amherst 

Image Provided bv awhitelev@nrc.umass.edu 
NSFForm1515 



Sampling zebrafish in a rice paddy in Bangladesh. 
Wild zebrafish occur in standing water like this rice 
paddy pond, or in streams with fast flowing currents. 


Credit: Richard Mayden, St. Louis University 


Adaptive evolution, changes in a species over time 
that enable it to adjust to an environment, has been 
studied extensively across many species. 
Technological advances have now made it possible 
to examine this evolution at the genomic level to 
address ecological evolution. Dr. Andrew Whiteley is 
using a fish species, zebrafish, in his research to 
determine how this species has adapted (using 
genes and physiology) to two different aquatic 
environments, fast moving rivers and calmer lakes. 
The zebrafish, which has over 75% of its genome 
sequenced, has been a useful vertebrate model 
organism to both developmental biologists and for 
biomedical research. 

The goal of this research is to provide a link 
between the fields of conservation genetics and 
evolutionary ecology. By advancing knowledge in 
both fields there will be an increased 
understanding of the relationship between 
phenotypic adaptation, gene expression, and 
genotype variations. This objective includes first, 
looking at gene frequencies for thousands of 
functional genes with newly developed genome 
technologies. A second step will involve 
examining expression for genes that appear to be 
important for adaptation to still or moving water. 
Ultimately this study will provide information on 
the evolutionary history of zebrafish in India and 
an initial understanding of conservation units in 
this fish species. 

Dr. Whiteley was supported from by an International 
Research Fellowship. Dr. Whiteley is working 
closely with Dr. Louis Bernatchez (knowledgeable 
in applying population genetics and genomic 
techniques to wild fish populations), Dr. Uttam 
Sarkar (Senior Scientist at the 
National Bureau of Fish Genetic Resources), Dr. M 
Arunachalam (Manonmaniam Sundaranar 
University), and Dr. Helga Guderley (expert in 
muscle physiology). Collaboration is between 


Image Provided bv awhitelev@nrc.umass.edu 
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the Universite Laval in Canada, and at the 
National Bureau of Fish Genetic 
Resources in India. 


Primary Strategic Outcome Goal: 

* International 
. Biology 

Secondary Strategic Outcome Goals: Learning 

* Postdoctoral Education and Fellowships 

* Professional and Career Development 

Does this highlight represent potentially transformative research?! f so, 

please explain why. For more information, see ReporttoCongress: Transformative 
Research attheNationalScienceFoundation April 1 6 , 2008 and ImportantNotice 
130: Transformative Research 

No 

What is the intellectual merit of this activity? 

This project aims to learn more about wild populations of an important laboratory 
model organism, the zebrafish (Danio rerio). Little is known about the distribution 
and evolutionary history of this species. 

What are the broader impacts of this activity? 

MeritReviewBroaderlmoactsCriterion:ReoresentativeActivities,Julv2007 

To what extent will it enhance the infrastructure for research and education, such as 
facilities, instrumentation, networks, and partnerships? 

This research establishes collaborations with researchers in India to 
facilitate integration of this important laboratory model with its natural 
history. 

O/D/OISE 2010 

NS F Award Numbers: 

0601864 
Award Title: 


Start Date: 

Expires: 

Awarded Amount to Date 
PI: 

Institution Name: 

R State Code: 

PE Codes: 


International Research Fellowship Program: Integrated 

Genomic Analysis of Neutral and Adaptive Variation in 

Natural Populations of the Zebrafish 

06 / 01/2007 

11 / 30/2009 

$ 160,397 

Andrew Whiteley, awhiteley@eco.umass.edu 
Whiteley Andrew 
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The quest of king penguins: orientation in a crowded environment 

Highlight ID: 21520, Version: AC/GPA 



King penguin colonies can 
stretch over several kilometres. 
Each breeding pair vigorously 
defends a square-meter 
residence area where they 
incubate an egg and brood a 
chick. When the chicks are 
several weeks old, they are left 
unattended in the groups of 
other chicks while the parents 
search for food. Here is the 
group of 10-12-month-old 
chicks next to breeding adults. 

Credit: Anna Nesterova 

Image Provided bv apnesterovaOqmail.com 

NSFForm1515 


Many of us at one point or another in life were 
lost in the crowd desperately searching for our 
family or friends, looking out for some familiar 
landmarks and not seeing anything because 
there were people everywhere. If only we knew 
how to search! King penguins ( Aptenodytes 
patagonicus) seem to solve problems like this 
quite easily and even without the help of cell 
phones. These birds have an extraordinary ability 
to locate a square-meter residence area among 
as many as 100,000 individuals on the flat 
beaches of sub-Antarctica. 

My research concerns animal navigation in 
"crowded" environments. The close proximity of 
large numbers of animals in colonial species can 
obstruct any locally available cues (e.g., visual, 
acoustic) making navigation especially 
challenging. Presently, I am trying to unravel the 
mechanisms of short-range navigation that allow 
king penguins to find their place in the colony in 
spite of all the odds. The flightless nature of 
penguins adds an additional layer of complexity 
for orientation in the colony, but at the same time 
it makes the situation similar to what we, 
humans, might face. 



Our research group has demonstrated that even 
at the age of 10 months, king penguin chicks 
have well-developed navigational abilities. They 
can come back to their exact place in the colony 
even if displaced a half kilometre away. In our 
field experiments we manipulate the availability of 
visual, acoustic and magnetic cues, and test the 
ability of penguin chicks to return to their place in 
the colony after a displacement. The results, 
obtained using high-precision GPS tracking, 
suggest that visual and acoustic cues are 
important for chick's navigation. Chicks pay 
special attention to the global features of the 
landscape (e.g. hills, lakes), but disregard local 
features (e.g., whale skeleton, rock formations) 
when homing. The sound of the colony can be a 



In order to be found and fed by 
the parents, chicks must 
remain at the specific place in 
the colony called rendezvous 
zone. However, the predators 
can drive chicks away from 
these zones. It is essential for 
chicks survival to navigate back 
to their place in the colony. 

Credit: Anna Nesterova 

Image Provided by apnesterova(a>qmail.com 

NSFForm1515 


determining factor in the chicks' initial 
orientation. On the other hand, the distortion of 
magnetic cues has no effect on chicks' homing 
ability. 

To address navigation questions in adult king 
penguins, we collaborate with our colleagues 
from CNRS at Strasbourg, France to analyze 
birds' movement patterns in the colony. We 
found that while younger birds (<8 years old) 
prefer to move when at least some visual cues 
are available, older birds can enter the colony 
even in complete darkness. How older birds 
manage to manoeuvre among thousands of 
individuals to a specific place in the colony 
during dark nights still remains a mystery. 

Primary Strategic Outcome Goal: 


• International 

• Biology 


Secondary Strategic Outcome Goals: Learning 


Chicks have their own games. 
Sliding down an old whale carcass 
is a popular activity. 

Credit: Anna Nesterova 

Image Provided 

bv apnesterova@qmail.com 

NSFForm1515 


• Postdoctoral Education and Fellowships 

• International Research Experiences for 
Undergraduate & Graduate Students 

Does this highlight represent potentially 
transformative research? If so, please explain 
why. For more information, see Reportto 
Congress:TransformativeResearchatthe National 
ScienceFoundation, April 1 6,2008 and Important 
Notice ? 30: TransfcrmaiiveResearch 

No 


What is the intellectual merit of this activity? 

The project assesses the use of visual landmarks, familiar routes, and olfactory cues 
and addresses an aspect of animal behaviour (short range spatial orientation) which 
has received relatively little scientific attention. 

What are the broader impacts of this activity ? Merit Re vie wBroaderlmoacts 
Criterion:RepresentativeActivities,Julv2007 

HoW well does the activity advance discovery and understanding while promoting 
teaching, training, and learning? 

This project promotes cross-fostering of ideas among scientists of different countries 
with unique resources and expertise. The principle investigator interacted closely 
with a graduate student in the field and also co-supervised undergraduate students at 
the research center. 




O/D/OISE 201 0 


NS F Award Numbers: 

0700939 

Award Title: 

Start Date: 

Expires: 

Awarded Amount to Date: 
PI: 

Institution Name: 

State Code: 

Codes: 


International Research Fellowship Program: The Quest of 
King Penguins: Orientation in a Crowded Environment 
06/01/2008 
08/31/2010 
$108,610 

Anna Nesterova, apnesterova@qmail.com 
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New insights into human evolution from ancient DNA 

Highlight ID: 21521, Version: AC/GPA 



Location and images of some of the Neandertal bones 
used for genetic analysis by Good and colleagues. 

Credit: Adrian Briggs and Johannes Krause, Max 
Planck Institute for Evolutionary Anthropology 

Image Provided by ieffrev.m.qood@qmail.com 

NSFForm1515 


Much of what is currently known about the 
evolutionary history of humans is based on 
analyses of fossilized remains and associated 
artifacts of ancient humans. However, despite 
fairly rich archeological and fossil records, 
relatively little is known about our closest 
relatives, the Neandertals. Remains of 
Neandertals begin to appear in the European 
fossil record about 400,000 years ago. In Europe, 
Neandertals began to come in to contact with 
modern humans migrating out of Africa 
approximately 50,000 years ago and ultimately 
disappeared around 30,000 years ago. Analysis 
of genetic information (i.e., DNA sequences) from 
Neandertals and other ancient human remains 
holds great promise for further unravelling the 
details of human evolutionary history, but retrieval 
of ancient DNA has proven to be very technically 
difficult. 



As an NSF International Research Fellow, Dr. 
Jeffrey Good has worked closely with pioneering 
ancient DNA researcher Dr. Svante Paabo, Dr. 
Johannes Krause, Dr. Adrian Briggs and other 
scientists at the Max Planck Institute for 
Evolutionary Anthropology (MPI-EVA) in Leipzig, 
Germany to recover genetic information from 
several ancient bones (70,000 - 30,000 years old) 
from parts of Europe and Asia. Dr. Good and 
colleagues found that Neandertals from Europe 
and western Asia had very low levels of genetic 
variation when compared to modern humans, 
indicating that Neandertals existed in relatively 
small populations prior to their extinction. A key 
component of this research was the development of a novel laboratory technique for 
retrieving highly degraded DNA from fossil bones. Dr. Adrian Briggs, who was a 
graduate student working with Dr. Good at the time of the research, developed this 
groundbreaking new method. 


Dr. Jeffrey Good. 

Credit: Jeffrey Good, University of Montana, Missoula 
Image Provided bv ieffrev.m.good@qmail.com NSF 
Form1515 


A follow-up study on a -40,000 year-old finger bone of a small child from southern 
Siberia provided an even more surprising result: examination of a small part of the 
genetic code retrieved from this Siberian bone revealed an unexpectedly high number 
of genetic differences when compared to both Neandertals and to present-day 



humans. Remains from Neandertals and archeological artifacts associated with 
modern humans from approximately the same time period have also been found in 
this region. There were no further skeletal clues to the child's identity but these 
genetic differences suggest that the bone may derive from a previously unknown 
population of humans. Ongoing genetic research at the MPI-EVA should help clarify 
the evolutionary relationship of this enigmatic bone relative to Neandertals and 
modern humans. 

Collectively, this research has provided considerable insights into human evolutionary 
history and has laid the groundwork for future genetic inquires into human evolution 
based on ancient DNA. Dr. Good and colleagues have published the results of this 
work in the high-profile scientific journals Cell, Science, and Nature. These works 
have also been highlighted in a diverse array of popular press forums including the 
New York Times, BBC News, and The Washington Post, sparking intense public 
interest in human evolution. 

Primary Strategic Outcome Goal: 

• International 

• Biology 

Secondary Strategic Outcome Goals: Learning 

• Postdoctoral Education and Fellowships 

Does this highlight represent potentially transformative research?lf so, please 
explain why. For more information, see ReporttoCongress:TransformativeResearch at 
theNationalScienceFoundation, April 1 6.2008 and ImportantNoticel 30: Transformative 
Research 

Yes 

This research has provided considerable insights into human evolutionary history and 
has laid the groundwork for future genetic inquires into human evolution based on 
ancient DNA. 

What is the intellectual merit of this activity? 

This took advantage of recent advances in primate comparative genomics and it 
addressed central theoretical issues concerning sexual selection and mating 
systems in great apes. This research will be of great benefit to those interested 
in the evolution of humans and other apes, especially in terms of male 
reproductive genes. 

What are the broader impacts of this activity ? Merit Re vie wBroaderlmpacts 
Criterion -.Representative Activities, Juiv2007 


Will the results be disseminated broadly to enhance scientific and technological 
understanding? 

Dr. Good and colleagues have published the results of this work in the high-profile 
scientific journals Cell, Science, and Nature. These works have also been highlighted 
in a diverse array of popular press forums including the New York Times, BBC News, 
and The Washington Post, sparking intense public interest in human evolution. 

O/D/OISE 201 0 


NS F Award Numbers: 

0754461 

Award Title: 

Start Date: 

Expires: 

Awarded Amount to Date: 
PI: 

Institution Name: 

Code: 

PE Codes: 


International Research Fellowship Program: Evolutionary 

Reproductive Genomics in Humans and Great Apes 
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10/31/2010 

$169,000 
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NSF Highlights: 2009 


Tool Use in the Turkana Basin 

Highlight ID: 18212, Version: AC/GPA 



One of Dr. Braun's foremen (Benjamin Sila) 
excavating part of a bovid skeleton. 

Credit: David Braun, University of Cape Town 

Permission Granted 


/ 
4 , 


Dr. Braun is writing field notes next to the equipment on 
the excavation site. 


The history of human evolution has been the 
subject of scientific inquiry for many years. 
Scientists have relied on archaeology (a field of 
science that studies historic and prehistoric 
peoples and their cultures through excavating 
and analyzing their remains, monuments, 
artifacts, and inscriptions) to aid them in 
understanding the timeline and the different 
developmental stages in human history. 

Dr. David Braun, through an International 
Research Fellowship, worked on an excavation 
for artifacts of and remains of early hominins 
(humans), who made and used stone tools in the 
northern part of the Turkana Basin in Kenya. As 
a result of this fellowship, Dr. Braun found a very 
dense concentration of bones and artifacts that 
are nearly two million years old. Through the 
analysis of these remains and artifacts, Dr. 

Braun determined that early humans: lived in the 
Turkana Basin about two million years ago; 
manufactured and transported stone tools; and 
used a riparian forest as a focus for their 
activities. 


credit: David Braun, university of cape Town Dr. Braun worked closely with experienced 

colleagues such as: Dr. Mzalendo Kbunjia (the 
main collaborator at the National Museums of 
Kenya, who hosted Dr. Braun's research project, provided the necessary facilities to 
store the specimens, and assisted in his analysis), and Dr. John Harris at Rutgers 
University, who has 30 years of experience in Turkana Basin. Additionally, Dr. Braun 
collaborated with several researchers at the University of Cape Town's Geology 
Department and Rutgers University. In collaboration with the Koobi Fora Research 
and Training Project, Dr. Braun conducted field trips for American, Kenyan, and 
South African students to the research sites. These field experiences positively 
influenced the students to the extent that some of them are now continuing their 
education, and working towards their degrees in archaeology. 




Finally, Dr. Braun reached out to the local communities around the excavation sites 
and met with village elders, educating them about the cultural and scientific importance 
of the sites. Dr. Braun published articles in local newspapers in Cape Town (The Cape 
Times) to raise awareness of his research and its impact on the field of human 
evolution. He also published several articles in scholarly journals such as 
Archaeological Science and the Journal of Human Evolution. 

Primary Strategic Outcome Goal: 

* International 

•Social, Behavioral, and Economic Research 
Secondary Strategic Outcome Goals: Learning 

* Postdoctoral Education and Fellowships 

Does this highlight represent potentially transformative research If so, please 
explain why. For more information, see ReporttoCongress:TransformativeResearch at 
theNationalScienceFoundation, April 1 6,2008 and ImportantNotice 1 30: Transformative 
Research 
No 

How well does the proposed activity broaden the participation of underrepresented 
groups (e.g., gender, ethnicity, disability, geographic, etc?) 

No 

What may be the benefits of the proposed activity to society? 

Yes 

In addition to increasing our knowledge in the field of archaeology, Dr. Braun forged 
strong collaborations with researchers from Kenya and South Africa. 

What is the intellectual merit of this activity? 

Dr. Braun's project provided new data on the behavior of early hominins. It developed 
a greater understanding of how and why early humans used stone tools and where 
they went to retreive stone to make tools. This is of vital importance to the study of 
evolution of human behavior because it shows that early humans that had a much 
more ape-like body form were already using stone tools to access meat and marrow. 
This is a major change from our last common ancestor with chimpanzees. This may 
suggest that stone artifacts were a major impetus behind the appearance of the 
modern human body form with long limbs and a larger brain. 

What are the broader impacts of this activity ? Merit Re vie wBroaderlmpacts 
Criterion :Representa tiveActivities, Julv200 7 

This work has increased our knowledge in the field of archaeology, and as indicated 
above, has been shared broadly via publication of Dr. Braun's results. 

In addition, Dr. Braun has done much to train future researchers and has assisted in 
their continuing education. 
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NSF Highlights: 2008 


Conservation and Management of African Elephants 

Highlight ID: 15158, Version: AC/GPA 

Dr. George Wittemyer was supported for 22 
months by a fellowship under the International 
Research Fellowship Program (IRFP) at 
Makerere University in Kampala, Uganda and the 
University of Copenhagen in Denmark. Dr. 
Wittemeyer's research was conducted in East 
Africa. 

Dr. Wittemyer's research addresses issues 
impacting the conservation and management of 
African elephants. With elephant populations 
increasing in some parts of Africa and declining 
in others, wildlife authorities face a range of 
conservation issues including human elephant 
conflict, ivory poaching, and increased 
confinement which amplifies the ecological role 
played by this species. These problems demand 
a multitude of solutions such as translocation, fencing and increased policing that 
can be more effectively implemented with science-based planning. Inspired by the 
difficulty of addressing these conservation concerns while securing the ecological 
requirements of this keystone species, Dr. Wittemyer's research uses genetic 
approaches to understand the factors influencing elephant population structure, 
movement and migration, and social interactions. His approach entails individual 
based research on a wild population of elephants in northern Kenya where he has 
identified over 700 elephants from ear patterns. 

Primary Strategic Outcome Goal: 

• Postdoctoral Education and Fellowships 
Secondary Strategic Outcome Goals: 

Does this highlight represent potentially transformative research ?lf so, please 
explain why. For more information, see ReporttoConciress:TransformativeResearch at 
theNationalScienceFoundation April 1 6.2008 and ImportantNoticel 30: Transformative 
Research 
Yes 

During the Fellowship, Dr. Wittemyer was trained and gained a depth of 
understanding in a new discipline in the biological sciences. This diversified his skills 
and advanced his ability to posit and address novel lines of research. 



Photo of Dr. George Wittemyer, 
working on elephant 
conservation in East Africa. 


Credit: Renee Kuriyan 

Permission Granted 



How well does the proposed activity broaden the participation of underrepresented 
groups (e.g., gender, ethnicity, disability, geographic, etc?) 

Yes 

This project developed and strengthened institutional collaborations between US 
science institutions, both University of California, Berkeley and University of Utah, 
and African science institutions, in Kenya both the Kenya Wildlife Service and Save 
the Elephants, and in Uganda Makerere University. 

What may be the benefits of the proposed activity to society? 

Yes 

The African elephant is a threatened species and is a poster child for conservation in 
Africa and Asia. As such, conservation and research activities on elephants are of 
great interest to the general public. 

How does this highlight address the strategic outcome goal(s) as described in the 
NSFStrategicPlan2006-201 1 ? 

This project developed and strengthened institutional collaborations between US 
science institutions, both University of California, Berkeley and University of Utah, and 
African science institutions, in Kenya both the Kenya Wildlife Service and Save the 
Elephants, and in Uganda Makerere University. In addition, the scientific integrity of a 
long-term field project in Africa was strengthened through Dr. Wittemyer's research. 

O/D/OISE 2008 

NS F Award Numbers: 

0502340 
Award Title: 

Start Date: 

Expires: 

Awarded Amount to Date: 

PI: 
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International Research Fellowship Program: Fine-Scaled 
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Evolution of Australian Acacia Thrips 

Highlight ID: 15182, Version: AC/GPA 


Dr. Jeremy Bono was supported from October 2004 through October 2006, by a 
fellowship under the International Research Fellowship Program (IRFP). This 
research was conducted at Simon Fraser University, Burnaby, B.C., Canada. As a 
result of Dr. Bono's participation in the IRFP, he is currently a faculty member, PERT 
Postdoctoral Fellow, Center for Insect Science, Dept, of Ecology and Evolutionary 
Biology, with the University of Arizona. 

The major objective of this research was to understand the genetic and ecological 
factors that promote the evolution of cooperative colony founding in one genus of 
Australian Acacia thrips (an insect from Australia). Progress toward this goal has 
been made in several ways. Two successful trips to Australia, where numerous 
collections of colony samples of four species that are central to the research were 
captured. These samples have been used for genetic analyses and also to discern 
some of the costs and benefits of cooperative colony founding for several species. 

Dr. Jeremy M. Bono organized a research project to take the initiative to learn more 
about the Australian Acacia. It was found that, at least in one species, cooperations 
during colony founding occurs among related females, though sometimes unrelated 
females are involved. The research identified two benefits of cooperative colony 
founding: individual foundress survival is higher for foundresses in groups than for 
solitary foundresses and cooperative colony founding leads to larger colony size, 
which provides an advantage in interactions with kleptoparasitic thrips ( stealing 
insects). Per capita reproduction declines with group size, indicating the possibility 
that some females in associations may behave altruistically by giving up 
reproduction. 

At least one species in the genus is strictly solitary. This is valuable to know for 
future comparative work. Dr. Bono states that he has gained experience with a 
number of new research techniques and analyses. He also has learned to work with 
microsatellites and analyse these data for population genetics. With these new 
skills, he can apply them to any other research that is possible. 

Primary Strategic Outcome Goal: 

* Postdoctoral Education and Fellowships 

Secondary Strategic Outcome Goals: 

Does this highlight represent potentially transformative resea rch?f so, please 
explain why. For more information, see ReDorttoCongress:TransformativeResearch at 
theNationalScienceFoundation, April 1 6,2008 and lmportantNotice130: Transformative 
Research 


Yes 


The broad question that this research addresses is what factors promote the 
evolution of cooperation. This is a long-standing question in evolutionary biology, and 
we now know that diverse sets of conditions can promote the evolution of cooperation 
in different organisms. One of the hopes is that by examining this question in a 
diverse array of organisms we can identify general prinicples that underlie all forms of 
cooperation, be it in human, thrips, or social amoebae. 

How well does the proposed activity broaden the participation of underrepresented 
groups (e.g., gender, ethnicity, disability, geographic, etc?) 

No 

What may be the benefits of the proposed activity to society? 

No 

How does this highlight address the strategic outcome goal(s) as described in the 
NSFStrategicPlan2006-201 1 ? 

This work on cooperation in Australian Acacia thrips is significant because it greatly 
expands our comparative database for understanding factors that promote the 
evolution of cooperative nesting. To date, most of the detailed studies on this problem 
have been conducted with the Hymenoptera (an order of insects). Dr. Bono was able 
to secure a second postdoctoral fellowship funded partly based on his previous track 
record. 

O/D/OISE 2008 
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Long-Term Effects of the Developmental Environment on the Immune 
Response in Ring-Necked Pheasants (Phasianus Colchicus) 

Highlight ID: 15325, Version: AC/GPA 


Dr. Jennifer Grindstaff was supported from June 2004 through August 2006, by a 
fellowship under the International Research Fellowship Program (IRFP) at Lund 
University, Lund Sweden. Dr. Grindstaff is currently an Assistant Professor with 
the Department of Zoology at Oklahoma State University. 

The primary objective of the project was to test the relative importance of the 
developmental environment, current condition, and MHC genotype on immune 
responses and parasite reisitance. The team initially proposed to conduct this 
research on a captive population of ring-necked pheasants (Phasianus colchicus). 
Ultimately, they conducted these studies on both pheasants and in a natural 
population of European starlings (Sturnus vulgaris). 

Dr. Grindstaffs research interests incorporated concepts and approaches from 
physiology, behavioral ecology and evolutionary biology. This integrative approach is 
a powerful way to understand both the proximate mechanisms and evolutionary 
consequences of parental investments in determining offspring performance. For 
example, although there is a wealth of knowledge on the proximate mechanisms of 
maternal antibody transmission, relatively little is known about individual differences 
within a species in maternal antibody transmission or differences among species in 
transmission. Conversely, maternal effects theory has historically neglected the 
mechanistic basis through which mothers influence offspring phenotype. One goal 
of her research is to explain differences among individuals and species in maternal 
provisioning by integrating the proxminate mechanisms of transmission with 
evolutionary theory on maternal effects. 

Primary Strategic Outcome Goal: 

• Postdoctoral Education and Fellowships 

Secondary Strategic Outcome Goals: 

Does this highlight represent potentially transformative resea rch?f so, please 
explain why. For more information, see ReporttoCongress:TransformativeResearch at 
theNationalScienceFoundation, April 1 6,2008 and lmportantNotice130: Transformative 
Research 

No 

How well does the proposed activity broaden the participation of underrepresented 
groups (e.g., gender, ethnicity, disability, geographic, etc?) 

No 


What may be the benefits of the proposed activity to society? 

Yes 

This research will provide critical information about the role of the developmental 
environment in adult phenotypic expression, especially the immune response. 

How does this highlight address the strategic outcome goal(s) as described in the 
NSFStrategicPlan2006-201 1 ? 

The primary objective of the project was to test the relative importance of the 
developmental environment, current condition, and MHC genotype on immune 
responses and parasite reisitance. The team initially proposed to conduct this 
research on a captive population of ring-necked pheasants (Phasianus colchicus). 
Ultimately, they conducted these studies on both pheasants and in a natural 
population of European starlings (Sturnus vulgaris). While Dr. Grindstaff was a 
postdoctoral fellow at Lund University, she also collaborated on two side projects. The 
research funds for the side projects were not obtained from NSF, however, if it were 
not for the IRFP, she would not have been a participant in these collaborations. Her 
participation as a postdoctoral fellow at Lund University, afforded her these additional 
opportunities. The objective of the projects were to determine if maternal immunization 
influences provisioning of eggs, offspring growth, and immune responses. 
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Sensory Integration and the Evolution of Multimodal Communication in 
Jumping Spiders 

Highlight ID: 15331, Version: AC/GPA 


Dr. Damian O. Elias was supported from September 2005 through June 2007 by a 
fellowship under the International Research Fellowship Program (IRFP). This 
fellowship has given Dr. Elias the experience and achievements necessary to accept 
a tenure-faculty appointment at a top level research institution, the University of 
California, Berkeley 

Jumping spiders have been extensively studied in the context of visual 
communication. They possess an unusually large pair of frontal eyes that confer a 
degree of spatio-visual acuity far above any other spider or insect. These visual 
specializations are associated with a complex repertoire of visual behaviors including 
prey capture, courtship, and agonistic displays. His research has demonstrated that 
surprisingly, in one extremely diverse genus in jumping spiders, males use complex 
multicomponent seismic (vibratory) signals coordinated with its visual signals. This 
discovery and more importantly the implications of this discovery are wide reaching. 
First of all, this research has implications into the study of diversity and the 
mechanisms leading to species evolution. Jumping spiders as a family are one of the 
most diverse animals in the world and the genus that he is working on, /Habronattus/, 
is one of the diverse genera in this family. His recent work has focused on 
understanding the mechanisms driving species evolution in the genus and in particular 
the role of sensory integration and multimodal communication. He has collected and 
examined over 50 species in the genus and using techniques from molecular, 
computational and evolutionary biology, he is in the process of studying patterns 
driving this extraordinary diversification. One of the most striking preliminary results 
from this study is that it appears that complexity between visual and vibratory signals 
are positively linked and that coordination of complex multimodal signals is associated 
with increases in species diversity. This link between sensory processing and species 
evolution will be an important contribution to worldwide research dealing with 
neurobiology, behavior, and evolution. 

Primary Strategic Outcome Goal: 

• Postdoctoral Education and Fellowships 

Secondary Strategic Outcome Goals: 

Does this highlight represent potentially transformative research If so, please 
explain why. For more information, see ReporttoConaress:TransformativeResearch at 
theNationalScienceFoundation, April 1 6,2008 and lmportantNotice130: Transformative 
Research 


No 


How well does the proposed activity broaden the participation of 
underrepresentedgroups (e.g., gender, ethnicity, disability, geographic, etc?) 

Yes 

Dr. Elias's work stemming from this project has provided opportunities for research 
and teaching in science, including people in underrepresented groups, and provided 
exposure to non-scientist members of the public. First of all, he endeavoured to 
include undergraduates in all aspects of his work. Many of these undergraduates 
were from underrepresented groups (including groups of African, Latino, and Middle 
Eastern descent). He attempted to involve undergraduates at all levels of scientific 
endeavor including experimental design, conducting experiments, and analyzing data 

What may be the benefits of the proposed activity to society? 

Yes 

Through these efforts Dr. Elias has continued to be involved with the scientific 
community and the general public through the lay media and outreach programs. 

How does this highlight address the strategic outcome goal(s) as described in the 
NSFStrategicPlan2006-201 1 ? 

International experiences for Graduate and Undergraduate students. 
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Tropical Forest Songbird Behavior and Movements in Human-Dominated 
Landscapes 

Highlight ID: 15562, Version: AC/GPA 



cocoa woodcreeper with 
radiotag shown 

Credit: Dr. John Withey 

Permission Granted 



Dr. Withey listens to the radio signals from one of the 
radiotagged individuals. 

Credit: Dr. John Withey 

Permission Granted 


Dr. John Withey was supported for 1 5 months in 
2007-2008 at the Smithsonian Tropical Research 
Institute (STRI) in Panama by a fellowship from 
the International Research Fellowship Program 
(IRFP), with funds from the Office of International 
Science and Engineering (OISE) Americas 
Program. Working with his host, Dr. Bill 
Laurance, his project was to study tropical forest 
songbird behavior and movements. He wanted to 
understand how the types of land use affect a 
bird's movement and behavior. Results of this 
work will provide conservation planners with 
critical information about the species that are 
expected to persist in highly fragmented, human 
dominated landscapes, and what types of land 
uses are compatible with providing habitats to 
birds. 

In Panama, forest has regrown in much of the 
former Canal Zone, even as human developments 
continue to spread. The lands surrounding these 
forests are used for different purposes, such as 
roads, buildings, housing developments, pasture, 
and agriculture. Dr. Withey tested how birds use 
both forested and non-forested lands, and how 
the type of land use affects the birds' movement 
and behavior. Understanding this topic is 
extremely important in planning for the protection 
of tropical bird species that use forested 
habitats. While large land reserves such as 
national parks will always be important to birds, 
tropical forests are being cut down at a rapid 
pace. As human populations grow, there will be 
more and more human-dominated landscapes in 
the world. Dr. Withey's research showed how 
birds use such lands, and how we may better 
plan land use to provide habitat for birds that 
depend on forests. 


Dr. Withey's approach was to capture, radiotag 
and release individual birds from three focal 
species: the dusky antbird, red-throated ant 




Male red-throated ant tanager held for banding. Credit: Dr. 
John Withey 
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tanager, and cocoa woodcreeper. The 
three species are different sizes and 
from different bird families, and are all 
relatively common in the forests of 
central Panama. Dr. Withey captured 
the birds in areas with both forested 
and non-forested habitat, as well as a 
variety of edge types.' For example, in 
these landscapes forest patches may 
be adjacent to a road, or buildings, or 
stands of Canal grass (a tall, invasive 
non-native species), or have natural 
gaps (forest openings) that create 
edges of forest with more-open habitat. 

Once a bird was radiotagged, Dr. Withey 
and his assistants tracked the bird's 
movements and recorded behavior and 
habitat types during 1 -hour observation 
periods. Using GIS software tocompile 
land cover and land use data, they 
compared relocations of each individual 


bird with the habitat and surrounding landscape. The resulting data was then analyzed 
to determine if each species has particular habitat preferences, if their behavior or 
movement rates are different in different habitats or edge types, and how large a gap or 
forest opening the birds will cross. 


Primary Strategic Outcome Goal: 

* Postdoctoral Education and Fellowships 

Secondary Strategic Outcome Goals: 

Does this highlight represent potentially transformative research ?lf so, please 
explain why. For more information, see ReporttoCongress:TransformativeResearch at 
theNationalScienceFoundation, April 1 6,2008 and lmportantNotice130: 

Transformative Research 

No 

How well does the proposed activity broaden the participation of 
underrepresented groups (e.g., gender, ethnicity, disability, geographic, etc?) 


No 



What may be the benefits of the proposed activity to society? 

Yes 

This fellowship gave a young U.S. researcher deeper insight and knowledge in this 
important area of research. When Dr. Withey returns to the U.S. and to academia, he 
will share that new knowledge in his work with students, in collaborating with other 
researchers in the field, and in outreach to the scientific community. In addition, 
results from his research will provide conservation planners with critical information 
about the species expected to persist in highly fragmented, human-dominated 
landscapes, and what types of land uses are compatible with providing habitat to 
birds. 

How does this highlight address the strategic outcome goal(s) as described in the 
NSFStrategicPlan2006-20 1 1 ? 

Dr. Withey will publish results of his research in peer-reviewed journals and present 
his findings at international conferences. He will also make presentations to 
Panamanian staff at the national parks in which he worked, as well as local 
conservation groups such as the Panama Audubon Society. He has benefited 
professionally from the connections made at STRI and has already submitted a 
research proposal to work on the effects of landscape change on bird communities in 
North America. He has also taken advantage of teaching and training opportunities in 
Panama by teaching a class on the principles of radiotelemetry for Panamanian 
college students. In addition to research experience, the IRFP provided Dr. Withey 
with invaluable experience in managing his own research budget and hiring and 
supervising assistants in a cross-cultural setting. 
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NMR Studies of Protein Folding: An Evolutionary Perspective 

Highlight ID: 15659, Version: AC/GPA 


Dr. Lesley H. Greene was supported from May 2000 through April 2003, by a 
fellowship under the International Research Fellowship Program (IRFP). Dr. Greene's 
research was conducted at the University of Oxford, considered to be one of the 
world's leading academic institutions. This research involved using nuclear magnetic 
resonance (NMR) to study the process of protein folding at atomic resolution. 

Along with her host collaborator, Dr. Christina Redfield, developed a novel way to view 
and study protein structures which challenges the traditional and thirty year old view. 
This fellowship also enabled her to learn advanced techniques to test ideas about 
protein folding which she first imagined as an undergraduate, developed as a graduate 
student and significantly advanced as a post-doctoral fellow at Oxford. Today as an 
Assistant Professor in the U.S.A., Dr. Greene is working with a multi-disciplinary 
group from the biological, physical, mathematical and computer sciences as well as 
her English collaborators in a continuation of this work. Together they aim to resolve 
the protein folding problem also known as the second-half of the genetic code which 
remains one of the most significant intellectual challenges in science today. This 
work has implications for research being done on many diseases such as 
Alzheimers, Parkinsons and Type II diabetes in understanding the mechanism of 
amyloid fibril formation. 


Primary Strategic Outcome Goal: 

• Postdoctoral Education and Fellowships 

Secondary Strategic Outcome Goals: 

Does this highlight represent potentially transformative research 71 f so, please 
explain why. For more information, see ReporttoConaress:TransformativeResearch at 
theNationalScienceFoundation April 1 6,2008 and lmportantNotice130: 
TransformativeResearch 

Yes 

In order to understand the determinant of a proteins topology, Dr. Greene in 
collaboration, modelled protein structures as network systems. This led to a novel 
way to view protein structures and a publication in the Journal of Molecular Biology 
(Greene and Higman, 2003). It also laid the foundations for a new methodology to 
rigorously analyze protein folds. During the course of this work, Dr. Greene also 
identified a group of proteins which share a common Greek-key topology but differ in 
secondary structure, sequence and function. Subsequent work following the 
fellowship period resulted in the identification of a conserved network which she 
believes is a key determinant of folding. Her results are published in Physics A 
(Higman and Greene, 2006). 


How well does the proposed activity broaden the participation of underrepresented 
groups (e.g., gender, ethnicity, disability, geographic, etc?) 

Yes 

Dr. Greene's scientific training and education at the University of Oxford has made her 
a better scientist, teacher and research mentor. This aids our countries' efforts to 
produce capable scientists here in the U.S.A. and better instruct our undergraduates 
in science, particularly biochemistry. Dr. Greene maintains that her past, present and 
future success will inspire more women to seek professions in the physical sciences in 
which they are underrepresented. 

What may be the benefits of the proposed activity to society? 

Yes 

One new avenue of research Dr. Greene is undertaking today seeks to identify the 
mechanism of amyloid fibril formation which underlies many diseases such as 
Alzheimers, Parkinsons and Type II diabetes. Her prior training during the fellowship 
period has well prepared her for this work. It is her hope that she will be able to 
significantly contribute to the prevention and treatment of these diseases which will 
help improve the health of the American people as well as the world community. 

How does this highlight address the strategic outcome goal(s) as described in the 
NSFStrategicPlan2006-201 1 ? 

Dr. Greene's work provides important insights which further the boundaries of our 
knowledge of the protein folding process. Her results also offer a new perspective in 
which to view and study protein structures which can stimulate the development of 
new theoretical and experimental approaches to advance a wide range of research 
efforts in the U.S. community. 
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Conservation and Management of African Elephants 

Highlight ID: 16715, Version: AC/GPA 

The African elephant is a threatened species 
and a poster child for conservation in Africa and 
Asia. In light of this situation, Dr. George 
Wittemyer used his grant under the International 
Research Fellowship Program (IRFP) to study 
the elephant population of Uganda, in 
collaboration with Dr. Silvester Nyakaana, 
Makerere University in Uganda, and with Dr. 

Flans Siegismund at Copenhagen University in 
Denmark. Based on genetic analyses, it was 
discovered that elephants form complex, 
hierarchical social organizations called nodes. 

For example, the older, dominant matriarchs are 
considered to be in a "grandparent" node. Though the elephants form these groups, 
there is often genetic mixing, as males cross from other groups to mate. It is 
important to protect the grandparent nodes in the social network, in order to ensure 
the integrity of the elephant population properties. These older members of the group 
are often targeted for ivory harvesting. Dr. Wittemyer found that there was rapid 
decline over periods of time caused by heavy poaching. By being able to track these 
various groups, this research helps people who work in conservation and wildlife 
management. By using these genetic approaches, they can understand the factors 
influencing the elephant population structure, movement and migration, and social 
interaction. The elephant populations are being conserved by relocating them, using 
fencing to direct their movements, and by increased policing. 

Primary Strategic Outcome Goal: 

• Postdoctoral Education and Fellowships 

Secondary Strategic Outcome Goals: Discovery 

* International 

Does this highlight represent potentially transformative research ?lf so, please 
explain why. For more information, see ReporttoConaress:TransformativeResearch at 
theNationalScienceFoundation, April 1 6,2008 and ImportantNotice 1 30 Transformative 
Research 

No 

How well does the proposed activity broaden the participation of underrepresented 
groups (e.g., gender, ethnicity, disability, geographic, etc?) 



George Wittemyer bolting a collar on an elephant. 
Credit: George Wittemyer 
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I 


No 



What may be the benefits of the proposed activity to society? 

No 

How does this highlight address the strategic outcome goal(s) as described in the 
NSFStrategicPlan2006-20 1 1 ? 

This fellowship allowed a young U.S. scientist to develop skills in a novel discipline. 
Because of this, he will expand the types and directions of research he and his 
students will carry out during his academic career. Specifically, he was trained in 
laboratory and analytical procedures in the field of population genetics. He has been 
awarded an adjunct professor position at Portland State University. 
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NSF Highlights: 2007 


Investigating the role of neural networks in mammalian working memory 

Highlight ID: 13293, Version: AC/GPA 

Working memory -- the ability to hold information 
for just long enough to complete a related task -- 
is evidenced in the brain of live monkeys by an 
elevated firing rate of some neurons in the 
pre-frontal cortex (PFC). One of the great 
unknowns in the cerebral cortex, however, is the 
actual distribution of functional synaptic 
connections and the role that various 
distributions might play. A greater understanding 
of these circuits in the PFC is necessary for 
understanding mechanisms underlying 
“intelligent” behavior in addition to several serious disorders, including Schizophrenia 
and Parkinson’s disease. 

Dr. Alexander Roxin was supported for 24 months by a fellowship under the 
International Research Fellowship Program (IRFP) at the University Rene 
Descartes in Paris, France to investigate the role of different synaptic distributions 
in a model of working memory by means of analytical and numerical methods. The 
IRFP is designed to provide young U.S. postdoctoral scientists and engineers with 
opportunities for joint research and the use of unique or complementary facilities, 
expertise and experimental conditions abroad. Dr. Roxin’s work was carried out at 
the Centre Nationale de la Recherche Scientifique (CNRS) Neurophysics and 
Physiology with his host Nicolas Brunei, a leading researcher in the field, and a 
handful of other collaborators. 

Dr. Roxin completed a systematic characterization of the emerging activity in a 
network of spiking neurons as a function of excitatory and inhibitory connections in 
the PFC. The work of Dr. Roxin and his collaborators has led to multiple 
conference poster presentations, invited lectures, and publications. The results 
published by the 

PI and his collaborators represent a large step forward in understanding the 
emergent activity in networks of large numbers of spiking neurons, and ultimately 
provide a necessary step in beginning to understand the neural correlate of 
intelligent animal behavior. Continued collaboration and future publications are 
expected from Dr. Roxin and his colleagues at the CNRS. 

Primary Strategic Outcome Goal: 

• Discovery: Foster research that will advance the frontiers of knowledge, 
emphasizing areas of greatest opportunity and potential benefit and establishing 
the nation as a global leader in fundamental and transformational science and 
engineering. 



Dr. Alexander Roxin in front of the Sacre Coeur 
Basilica in Paris, France. 

Credit: Alexander Roxin 
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Secondary Strategic Outcome Goals: Learning 

• Learning: Cultivate a world-class, broadly inclusive science and engineering 
workforce, and expand the scientific literacy of all citizens. 

Does this highlight represent transformative research? 

No 


How does this highlight address the strategic outcome goal(s) as described in the 
NSFStrategicPlan2006-20 1 1 ? 

This award gave an opportunity to a postdoctoral researcher to foster research that 
will advance the frontiers of knowledge, as stated in the "Discovery" strategic 
outcome goal. 
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NSF Highlights: 2007 

Duetting in the New Zealand Kokako 

Highlight ID: 13295, Version: AC/GPA 
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When Dr. Laura Molles began investigating the 
bird song of New Zealand’s endangered, endemic 
kokako, she never imagined her research would 
lead directly to a unique conservation application. 
Most of us think of bird song as something males 
perform to woo prospective mates; in northern 
temperate regions of the world, including the 
United States, this stereotype tends to hold true. In 
many tropical and southern hemisphere species 
however, such as the kokako, male and female 
partners sing in striking coordination. 


An NSF International Research Fellowship 
allowed Dr. Molles to investigate the structure 
and functions of this style of singing, called 
duetting, in collaboration with Dr. Joe Waas at 
the University of Waikato in New Zealand. Their 
research suggested that kokako duets are, at 
least in part, a cooperative effort on the part of 
mated pairs that likely serves a variety of 
communicative functions, including the defense 
of large, year-round territories and pair bonds. 
Kokako duets, besides being extremely beautiful, are incredibly complex and unlike 
any other described bird song. A typical kokako duet lasts 30 to 45 seconds and 
unfolds slowly, with lengthy but carefully measured pauses. Males and females tend 
to sing particular portions of the song, but some pairs are able to swap singing roles 
from song to song without a loss of coordination. 


Dr. Laura Molles and New Zealand’s endangered 
kokako. 

Credit: Dr. Laura Molles 
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Dr. Molles performed a series of playback experiments designed to discover what it is 
about these duets that makes them more effective territorial signals than solo songs. 
Her results showed that kokako differentiate strongly between one and two sources of 
sound, and suggested that they may be capable of discriminating female from male 
singers despite the great flexibility in male and female contributions. Her “pure 
science” investigation of kokako song furthermore led directly to a unique 
conservation application: the use of song playback as an “anchor” for kokako during 
reintroduction to restored habitats. This application has proven to be an exciting and 
very rewarding marriage of behavioral ecology and conservation. 


In addition to allowing Dr. Molles to undertake a challenging project of her own 
design, the NSF fellowship gave her the opportunity expose other U.S. students and 
scientists to New Zealand’s successful model of combining pure and applied science. 
During her fellowship Dr. Molles employed two field assistants from the United States 



and helped teach an undergraduate course in conservation biology for a University of 
Wisconsin study abroad program. 

Due in part to the strength of her research program that combined original, 
question-driven science with real-world conservation, Dr. Molles obtained a faculty 
position as a lecturer at New Zealand’s Lincoln University. She teaches courses in 
ecology and conservation, and is supervising graduate students working with 
penguins, parrots and songbirds. The collaborations established during her 
fellowship - not only with her host scientist and institution, but also with the 
Department of Conservation and other research institutions in New Zealand - 
continue to benefit her career as she expands her kokako research and moves on 
to new projects. 

Primary Strategic Outcome Goal: 

* Discovery: Foster research that will advance the frontiers of knowledge, 
emphasizing areas of greatest opportunity and potential benefit and establishing 
the nation as a global leader in fundamental and transformational science and 
engineering. 

Secondary Strategic Outcome Goals: Learning 

• Learning: Cultivate a world-class, broadly inclusive science and engineering 
workforce, and expand the scientific literacy of all citizens. 

Does this highlight represent transformative research? 

No 

How does this highlight address the strategic outcome goal(s) as described in the 
NSFStrategicPlan2006-201 1 ? 

This award gave an opportunity to a postdoctoral researcher to foster research 
that will advance the frontiers of knowledge, as stated in the "Discovery" 
strategic outcome goal. 
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NSF Highlights: 2006 


East African Paleoclimate from Ancient Lake Clays: Linking Ecosystem 
Change with Hominid Evolution 

Highlight ID: 11302, Version: AC/GPA 
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Diatoms (Photosynthesizing plankton)and phytoliths 
(plant parts made of silica) from Ngorongoro wetland 
sediments. In some circumstances, the silica in these 
biological compoments interacts with saline waters to 
form clay minerals that can reveal information on the 
chemistry of the waters in which they form. 


Credit: Daniel M. Deocampo 
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View across Olduvai Gorge, Tanzania, at the 3rd Fault. 
The layer-cake stratigraphy is valuable for regional 
paleoenvironmental reconstruction, and reconstructing 
environmental change in this paleo-lake basin over the 
past couple million years. 

Credit: Daniel M. Deocampo 
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Dr. Daniel Deocampo was supported for 13 
months by a fellowship under the International 
Research Fellowship Program (IRFP) to study at 
the Department of Mineralogy at the Natural 
History Museum in London, United Kingdom. The 
IRFP is designed to launch young U.S. 
postdoctoral scientists and engineers into 
becoming globally-engaged. 

The main purpose of his project was to continue 
to develop the technique of reconstructing 
ancient climate changes through analyzing the 
chemistry of nano-scale clay minerals that 
formed from ancient lake waters. This project has 
been field-intensive, with work carried out in 
several lake basins of northern Tanzania and 
Kenya, including several that are important to our 
understanding of human evolution. A detailed 
reconstruction was made for the paleo-lake at 
Olduvai Gorge, Tanzania, and provided a record 
of environmental fluctuations around 1 .8 million 
years ago in East Africa, that has provided a 
basis for understanding interactions between 
environmental change at that time, human 
evolution, and the rise of technology among our 
ancestors. 

Understanding modern environments has been an 
important part of reconstructing the past, and 
Deocampo's related work has produced new 
understandings in environmental chemistry. This 
has contributed to the scientific basis needed in 
sub-Saharan Africa for balancing the competing 
needs of indigenous communities, wildlife, and 
economic development, in sensitive environments 
such as the Ngorongoro Crater, Tanzania, a 
World Heritage Site. This research focus has led 
Deocampo to a new effort to use similar 
approaches in studying lake sediment in western 
North America, such as in the Barstow Basin, 


Ngorongoro Crater, Tanzania, where studies of the 
modern environment can not only be used for 
environmental protection, but also to help understand 
sedimentary and geochemical records of similar 
ancient environments. 


Credit: Daniel M. Deocampo 
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California, and the Great Basin lakes such as 
Mono, Pyramid, and Walker Lakes. 

This line of research has also led to new areas of 
research on formation of clays and other 
minerals in modern landscapes. Deocampo is 
now studying the development of soils in the 
Coast Range of northern California, and how it 
impacts the origin and fate of mercury, a major 
environmental hazard in the region. He is also 
studying the mineralogical forms and 
geochemical behavior of lead in urban 
Sacramento, using his interdisciplinary 
background and international collaborations to 
find solutions to contemporary American 
environmental problems. 


The active volcano Oldoinyo Lengai, as viewed from 
near the top of the East African Rift escarpment. Ash 
from volcanoes such as this affect the chemistry of 
water and sediment throughout the region, and provide 
datable time-marker horizons in lake sediment. 


Credit: Daniel M. Deocampo 
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Deocampo with Masaai guides and Richard Hay, 
geologist for Mary Leakey during the 60's and 70's at 
Olduvai Gorge. 


Credit: Daniel M. Deocampo 
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In all this work, the international collaborative 
relationships Deocampo formed in London, and 
elsewhere in Europe while on fellowship there, 
have been critical in his professional development 
and scientific progress. The mineralogical 
techniques he was exposed to in London are 
state-of-the-art, and he brings those experiences 
and perspectives to his new studies in the U.S. 

He maintains an active collaborative relationship 
with Dr. Javier Cuadros, of the Natural History 
Museum, who is an expert on the detailed 
crystal structure of clays and other minerals. 
Cuadros visited Deocampo in Sacramento during 
June, 2005, to continue their collaborative work 
on a project representing a significant advance in 
understanding the chemistry of these minerals, 
and how they interact with lake and soil waters. 

In this effort, Deocampo is leading the team of 
British, Spanish, and Polish scientists, and they 
plan to submit a joint manuscript reporting these 
results in the coming months. As a result of their 
meeting, Cuadros has agreed to continue 
collaborating, to bring his expertise to 



Deocampo's new studies of mercury interaction with minerals in streams and soils 
of northern California. 

Primary Goal Indicators: 

•Collaborations 

Secondary Goal Indicators: 


Other Indicators (Is this work transformative or multidisciplinary?): 

This work involves multidisciplinary research. 

This project involves chemistry and geology, as well as increasing our understanding 
of human evolution and the rise of technology. 

This work is notable because 

This international research experience made Dr. Deocampo more marketable in his 
search for a tenure track position. In addition, he has continued his collaboration with 
his foreign host, which has allowed his research to expand and flourish. 
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Arctic Cetaceans: Indicators of Climate Change 
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Air 





Credit: Dr. Kristin Laidre 
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A bowhead whale in Disko Bay, West Greenland. This 
is one of the three Arctic whale species that occupies 
habitat above the Arctic circle and is highly vulnerable 
to climate change. 

Credit: Dr. Kristin Laidre 
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Dr. Kristin Laidre was supported for 22 months 
by a fellowship under the international Research 
Fellowship Program (IRFP) at the Greenland 
Institute of Natural Resources in Nuuk, 
Greenland. The IRFP is designed to launch 
young U.S. postdoctoral scientists and 
engineers into becoming globally-engaged. 

Dr. Laidre's research addresses questions 
concerning the vulnerability of arctic cetaceans 
to climate change. The Arctic is currently 
experiencing dramatic changes in sea ice 
characteristics and marine productivity, which 
will have cascading effects on Arctic food webs. 
Three species of cetaceans (the narwhal, 
beluga, and bowhead whale) inhabit arctic 
waters of West Greenland and are ideal 
indicator organisms for monitoring ongoing 
biophysical changes impacted by a warming 
climate. Arctic cetaceans' seasonal movements, 
distribution, resource selection, and life history 
parameters are tightly linked with changes in the 
arctic environment, making them both 
vulnerable to climate alterations and good 
indicators of cumulative changes. 

The main purpose of the project has been to 
contrast arctic species inhabiting different 
ecological niches with different life history 
strategies to support broader inferences 
regarding the effects of climate changes in 
different habitats of the high Arctic. Dr. Laidre 
has quantified the trends in sea ice and primary 
production in focal areas using satellite 
telemetry data on whale movements and diving 
behavior and remotely-sensed environmental 
data. This information has been used to 
examine resource selection and differential 
vulnerability based on quantitative spatial 
modeling techniques. Results of this work 


have facilitated the understanding of the potential effects of climate change on high 
arctic top predators and the sustainability of their exploitation by Inuit communities, 
linking scientific discovery to societal benefit. 

The host institute, the Greenland Institute of Natural Resources (GINR) in Nuuk, 
Greenland, is the Greenland Home Rule Government's center for nature research 
focusing on conservation, climate change and human impacts, biological diversity, 
and sustainable use of living resources. Dr. Laidre's research has been in 
collaboration with senior scientist Dr. Mads Peter Heide-Jorgensen at GINR. 

In addition to several peer reviewed scientific manuscripts and a popular book 
published during the fellowship, Dr. Laidre's research has extended into an 
international review paper together with several top arctic ecologists developing a 
quantitative index to rank the vulnerability of arctic marine mammals to climate 
change. In all this work, the international collaborative relationships Dr. Laidre formed 
in Denmark/Greenland, and elsewhere in Europe while on fellowship there, have been 
critical in her professional development and scientific progress. Her work has 
elucidated broad cetacean resource selection relationships and documented 
biocomplexity associated with changing climate. 

Primary Goal Indicators: 

•Global S&E workforce 


Secondary Goal Indicators: 

•Collaborations 

Other Indicators (Is this work transformative or multidisciplinary?): 

This work involves multidisciplinary research. 

This research is of importance those working on issues of climate change as well as 
to biologists who study diversity and conservation. 

This work is notable because 

This award provided a young U.S. researcher with a unique opportunity to work in 
Greenland with top arctic ecologists. She has already published results and has 
formed relationships that will be important throughout her career. 
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Algal Biodiversity and Landscape Patterns in Tropical Streams 

Highlight ID: 10885, Version: AC/GPA 



In the neotropics, where diversity of larger 
organisms is not well understood, even less is 
known about the microscopic organisms 
inhabiting the rainforests. Diatoms, a group of 
microscopic algae, are an important part of the 
"primary producer" community in streams and 
rivers in these forests. Species of diatoms often 
live within narrow environmental conditions, 
making them important indicators of the aquatic 
and terrestrial environments, e.g., light, nutrient 
levels, and salinity. Understanding the diversity 
and role of diatoms in streams is important to 
comprehending how rapid deforestation in the neotropics is affecting associated 
stream ecosystems. 


Credit: Dr. Rebecca Bixby 
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Dr. Rebecca J. Bixby, supported under the International Research Fellowship 
Program (IRFP), has studied the biodiversity and landscape patterns of diatoms in 
lowland neotropical streams. These streams are located at La Selva Biological 
Station, in the lowlands of Costa Ricas northern Caribbean slope. Dr. Bixby, in 
collaboration with Costa Rican and American scientists and students, has collected 
over 800 diatom samples from these rainforest streams over a two-year period. 


Dr. Bixby estimates that nearly 30% of the species collected are new to science and 
that many more have been described only from the neotropics. One notable example 
is the discovery of a new diatom species which is dominant in high-solute streams. 
Seminavis silvatropicalis sp. nov. is the first freshwater species described in a genus 
which formerly only contained marine and estuarine species (Bixby and Wydrzycka, 
submitted). Since the transition from a marine to freshwater habitat (or vice versa) is 
considered a major evolutionary step, the discovery of this freshwater diatom is an 
interesting find. 

In addition to illuminating the biodiversity of rainforest diatoms, these collections have 
been used to show relationships between species community changes and 
chemical/physical parameters. Distinct shifts in species composition in response to 
solute, light, and pH levels have been observed. This information can be used in 
environmental monitoring, giving a clear indication of ecosystem health. 

Primary Goal Indicators: 

* Global S&E workforce 



Secondary Goal Indicators: 

• Collaborations 

Other Indicators (Is this work transformative or multidisciplinary?): 

No other indicators apply. 

This work is notable because 

This fellowship allowed a young, female researcher to work outside of the United 
States at a notable research site in Costa Rica where a large number of international 
researchers are stationed. The fellowship gave her the opportunity to establish herself 
in an exciting field, allowing her to collaborate and forge relationships with 
internationally-respected biologists from other countries. 
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Synthesis of Novel Magnetic Nanoparticles 

Highlight ID: 9354, Version: AC/GPA 

Method tor forming isolated nanopaiticles in 
polymer templates 

Most current anticancer treatments destroy 
cancer cells by stopping them from growing or 
multiplying. In the process, healthy cells can be 
harmed and cause damaging side effects to 
cancer patients. Dr. Linda Harris, a synthetic 
polymer chemist from Virginia Tech and recipient 
of a prestigious NSF International Research 
Fellowship Program (IRFP) award, is working 
with world-renowned scientists at the University 
of Western Australia to provide doctors and 
patients with a better alternative that could use 
advanced nanoparticles to deliver drugs directly 
to a cancerous tumor without harming other parts 
of the body. Their research is an interdisciplinary 
collaboration to specially prepare optimized iron 
oxide nanoparticles. Without preventive 
measures, high surface energies observed in 
nanoparticles leads to aggregation and 
precipitation from solution. One common method 
to prevent this adverse effect is to coat the 
materials with polymer surfactants after they are 
prepared. However, characterization of these 
solutions often indicates that some aggregation 
is still observed. Harris aims to prepare polymers 
that are capable of forming spherical micellar 
structures in solution. The formation of the iron oxide nanoparticle inside the polymer 
template will prevent interaction of the particles which leads to aggregation and thus 
create well defined isolated nanoparticles. These advanced nanoparticles have 
potential electronic and biomedical applications such as magnetic field directed drug 
delivery, magnetic separations, and magnetic resonance imagining (MRI) contrast 
agents. This development of magnetic particles allows drugs to be guided through the 
body to target specific disease sites. Toxic drugs placed in an ampoule or enclosed 
capsule could go directly to a cancerous tumor and destroy cancer cells without 
harming other parts of the body. The ampoule would then pass out of the body after 
delivering the drug. 

Primary Goal Indicators: 

•Collaborations 

Secondary Goal Indicators: 

•Global S&E workforce 



These two images illustrate the method to form 
well-defined iron oxide nanoparticles inside of reverse 
micelle (nanoreactors) polymer templates. The diblock 
copolymer is poly(butyleneoxide)-b-poly(ethylene 
oxide) and was synthesized by living anionic 
polymerization. 
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Polymer Templates for Iron Oxide 
Nanoparticle Formation 


Diblock polym 
Hydrophobic Hydrophilic 



Image 2. 
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Other Indicators (Is this work transformative or multidisciplinary?): 

No other indicators apply. 

This work is notable because 

This international research experience gives Dr. Harris the opportunity to work with 
physicist Tim St Pierre and other researchers in Australia. Each side contributes their 
unique expertise and experience to the project. Dr. Harris brings experience in 
polymers; her colleagues at the University of Western Australia, biomagnetics. St 
Pierre works in the field of the structure and magnetism of nanoscale iron oxides, 
particularly those found in biological systems. His most significant contributions lie in 
his multidisciplinary approach, which has enabled an unprecedented systematic 
investigation into the relationships between structure and magnetic properties of oxide 
nanoparticles. 
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Bone Strength and Fracture Healing 

Highlight ID: 9355, Version: AC/GPA 

Dr. Sandra J. Shefelbine was awarded an International Research Fellowship to 
go to Ulm, Germany to work with Dr. Lutz Claes at the University of Ulm's 
Institute of Orthopaedic Research and Biomechanics. This institute is one of the 
leading centers world-wide for research on fracture healing. Their research in 
Ulm focuses particularly on the influence of mechanical loads on fracture 
healing and the mechanical properties of the bone fracture as it heals. The 
institute provides a unique interdisciplinary atmosphere as doctors, 
veterinarians, physicists, cell biologists, and engineers work together on a single 
project. 

This combination of disciplines creates an excellent environment to develop 
new techniques and methods. This particular research project focused on 
combining imaging and computer modelling techniques in order to determine 
how well a fracture is healing. Such a method could be a useful tool for doctors 
to determine how much longer a fractured bone must be stabilized, if additional 
stabilization is needed, or what the effects of new treatments are on the healing 
process. 

Fracture healing studies examine the effects of surgical procedures, drugs, 
mechanical loading, and other treatments on the bone healing process in order 
to accelerate recovery time and bone quality. Most fracture healing studies use 
animal models to measure the effects of the treatments. Throughout the healing 
process, the fractured bones are explanted, and the strength of the bones is 
determined using mechanical tests. 

With improvements in computed tomographic (CT) imaging technology (similar 
to a 3 dimensional x-ray), images of bones down to micron resolution are 
possible. These images provide information on the shape as well as the 
mineral density of the bone. The mineral density is proportional to the material 
properties, in particular the elastic modulus (a high density indicates a large 
elastic modulus and, therefore, a stiffer the bone). Finite element analysis 
(FEA) uses a mathematical representation of a structure to determine of the 
force and displacement at any point within the structure. As input, finite element 
models require the geometry and the material properties of the structure. 
Therefore, CT image data is aptly suited as input for finite element 
models. The objective of this research was to combine CT imaging with finite 
element analysis in order to predict the strength of a healing bone. 

Whereas mechanical testing requires explantation of the bone and usually 
provides only a single value as an estimate for callus quality, the combination 
of CT images and FEA has the potential to predict multiple mechanical 
properties (bending and torsion in multiple planes) in living animals. Predicting 
mechanical properties from CT images in combination with FEA could prove to 


be a useful tool in fracture healing studies, supplementing or replacing 
biomechanical testing. 

Dr. Shefelbine's experience in Ulm has been both academically and personally 
an invaluable experience. Ulm is particularly well-known for its excellence in 
experimental research, which was the perfect complement to Shefelbine's 
primarily theoretical graduate work. Both the personnel and equipment 
resources in Ulm are outstanding. She has learned numerous experimental 
techniques from experts in the field, using excellent equipment. A two-year post 
doctoral fellowship gave her additional publications, confidence, ideas, and 
direction so that she is now prepared to lead her own research group. In 
addition to academically preparing her, she has made innumerable contacts 
and can foresee potential European collaborations. 

Primary Goal Indicators: 

• Collaborations 

Secondary Goal Indicators: 

• Global S&E workforce 

Other Indicators (Is this work transformative or multidisciplinary?): 

No other indicators apply. 

This work is notable because 

This international experience helped prepare a young researcher for a career 
in academia which will include long-term research collaborations with 
scientists from around the world. 
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Appendix B: Non-Response Bias Study 


This appendix describes the non-response bias study conducted for the applicant survey. First, it 
describes the two types of missing data. Next, it describes the analytic approach for addressing each 
type of non-response. 


Types of Missing Data 


There are two types of missing data that can arise in a survey, even after repeated attempts to 
collect data: (1) unit non-response, and (2) item non-response. The approach to dealing with each of 
these in this study is described below. 

Unit Non-Response 

Unit non-response occurs when an entire data instrument is not received from a potential 
respondent. Because non-response was greater than 10 percent, available data were used to 
explore whether results might be affected by non-response bias. Large differences in the response 
rates for subgroups could indicate that potential biases may exist. 1 For example, if the response rate 
from women was very low, and women were less likely to belong to the treatment group, then any 
observed difference in the outcomes between the treatment and comparison groups could 
potentially result in a biased estimate of the impact of the treatment. Exhibit B.l shows the 
variables used in this analysis. Exhibits B.2 to B.4 show the response rates by groups, suggesting that 
there were differences in response rates by subgroups. 


Exhibit B.l: Variables Used in Non-Response Analysis 


1 Variable 

Variable Name 

Values 

N (%) 

Gender 

cr_gender_nrbias 

0=Male 

993 (61.00) 



l=Female 

635 (39.00) 

Under-Represented Minority 

Cr_URM_nrbias 

0=Non-Minority 

1405 (86.30) 



l=Minority 

100 ( 6.14) 



2=Missing 

123 ( 7.56) 

Disability Status 

cr_hdcap_nrbias 

0=Non-Disabled 

1415 (86.92) 



l=Disabled 

9 ( 0.55) 



2=Missing 

204(12.53) 

Cohort 

nYear_App_group 

1=1991-2000 

510(31.33) 



2=2001-2005 

586 (36.00) 



3=2006+ 

532 (32.68) 

Award Status 

nfinalawdfin 

0=Non-Awardee 

1064(65.36) 



l=Awardee 

564 (34.64) 

Proposal Score 

Finalscore 

1-5 

Min=1.33, Max=5 
Mean=3.97, Std=0.66 
N=1628 


1 Note that a large non-response rate does not necessarily create bias. For example, if the non-respondents 
were similar across the treatment and comparison groups, then the impact estimate would not be biased 
necessarily; rather, any effect of the program could not be generalized to the non-respondents (i.e. it 
would create an external validity problem but not necessarily an internal validity issue). 
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Examining Response Rates Overall 


Exhibit B.2: Response Rates by Subgroups of Interest 


Characteristic 


% Responding (n) 


p-value 

Gender 

Males 

Females 

Missing 

0.0475* 


61.93 (615) 

66.77 (424) 

- 

Under- 

Non-URM 

URM 

Missing 


Represented 

Minority 

68.11 (957) 

72 (72) 

8.13 (10) 

<0.0001** 

Disability Status 

Non-Disabled 

Disabled 

Missing 

<.0001** 

72.16 (1021) 

44.44 (4) 

6.86 (14) 

Award Status 

Non-Awardee 

Awardee 

Missing 

<.0001** 

54.7 (582) 

81.03 (457) 

- 

Application 

Cohort 

1991-2000 

61.18 (312) 

2001-2005 

58.53 (343) 

2006+ 

72.18 (384) 

<.0001** 

Average Proposal 

Non-Responders 

Responders 

Missing 

<.0001** T 

Score 

3.78 (589) 

4.08 (1039) 

- 


NOTES: 

** p-value <0.01 

Unless otherwise specified p-values are from a 2X2 chi-square test of the null hypothesis of no association between 
participation and the characteristic of interest (1 degree of freedom). 

F p-value is from the Fisher's Exact Test 
T p-value is from the Student's T-Test 
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Examining Response Rates by Award Status 


Exhibit B.3: Response Rates by Subgroups of Interest Controlling for Award Status 




% Responding 


p-values 


Characteristic 

(n) 

2X2 Chi Square 

Award Status CMH 

Gender 

Non-Awardee 

Male 

53.24(345) 

0.2529 




Female 

56.97 (237) 


0.0376* 

Awardee 

Male 

78.26 (270) 

0.0354* 



Female 

85.39 (187) 



URM 

Non-Awardee 

Non-URM 

59.60 (540) 





URM 

61.67 (37) 

<0.0001** 




Missing 

5.10 (5) 



<0.0001** 

Awardee 

Non-URM 

83.57 (417) 




URM 

87.5 (35) 

<0.0001** 




Missing 

20.00(5) 




Disabled 

Non-Awardee 

Not Disabled 

64.49 (572) 





Disabled 

42.86 (3) 

<0.0001** 




Missing 

4.17 (7) 



<0.0001** 

Awardee 

Not Disabled 

85.04 (449) 




Disabled 

50(1) 

<0.0001** 




Missing 

20.59 (7) 




Application Cohort 

Non-Awardee 

1991-2000 

48.29 (141) 





2001-2005 

50 (202) 

<0.0001** 




2006+ 

64.95 (239) 



<0.0001** 

Awardee 

1991-2000 

78.44(171) 




2001-2005 

77.47 (141) 

0.016* 




2006+ 

88.41 (145) 





Mean Score 3 






Non- 

Mean Score 3 




Proposal Score 

Responders 

Responders 


p-value 


Non-Awardees 

3.62 

3.74 


0.0006 1 


Awardees 

4.49 

4.49 


0.9929 1 



Missing values are assigned their own level. 

** p-value <0.01 

Unless otherwise specified p-values are from a 2X2 chi-square test of the null hypothesis of no association between 
participation and the characteristic of interest (1 degree of freedom). 

a Mean score is the average of all individual scores assigned by independent reviewers. Scores ranged from 1 (Poor) to 
5 (Excellent) and each proposal was typically scored by 3 reviewers. 

T p-value is from the Student's T-Test 

The column labeled "Award Status CMH" shows the p-values from Cochran-Mantel-Haenszel tests of the null 
hypothesis of no common participation effect across award status. 
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Examining Response Rates by Application Cohort 


Exhibit B.4: Response Rates by Subgroups of Interest Controlling for Application Cohort 






p-values 



% Responding 


Application Cohort 

Characteristic 

(n) 

2X2 Chi Square 

CMH 

Gender 

1991-2000 

Male 

58.81 (197) 

0.1286 



Female 

65.71 (115) 


2001-2005 

Male 

59.43 (208) 

0.5918 

0.0676 


Female 

57.20 (135) 

2006+ 

Male 

68.18 (210) 

0.0158* 



Female 

77.68(174) 


URM 

1991-2000 

Non-URM 

62.02 (285) 

<0.0001** 



URM 

75 (21) 



Missing 

9.52 (6) 



2001-2005 

Non-URM 

63.56 (314) 

<0.0001** 



URM 

66.67 (26) 

<0.0001** 


Missing 

5.66 (3) 



2006+ 

Non-URM 

72.76 (358) 




URM 

75.76 (25) 

0.0025** 



Missing 

14.29 (1) 



Disabled 

1991-2000 

Not Disabled 

67.84 (308) 




Disabled 

66.67 (2) 

<0.0001** 



Missing 

3.77(2) 



2001-2005 

Not Disabled 

68.29 (336) 

<0.0001** 

<0.0001** 


Disabled 

25(1) 


Missing 

6.67 (6) 



2006+ 

Not Disabled 

80.38 (377) 

<0.0001** 



Disabled 

50(1) 



Missing 

9.84 (6) 



Award Status 

1991-2000 

Non-Awardee 

48.29 (141) 

<0.0001** 



Awardee 

78.44 (171) 


2001-2005 

Non-Awardee 

50 (202) 

<0.0001** 

<0.0001** 


Awardee 

77.47 (141) 

2006+ 

Non-Awardee 

64.95 (239) 

<0.0001** 



Awardee 

88.41 (145) 



Mean Score 3 





Non- 

Mean Score 3 



Proposal Score 

Responders 

Responders 


p-value 

1991-2000 

3.83 

4.13 


ww 

<0.0001 

2001-2005 

3.73 

4.07 


<0.0001** 

2006+ 

3.79 

4.04 


<0.0001** 


Missing values are assigned their own level. 

** p-value <0.01 

Unless otherwise specified p-values are from a 2X2 chi-square test of the null hypothesis of no association 
between participation and the characteristic of interest (1 degree of freedom). 

a Mean score is the average of all individual scores assigned by independent reviewers. Scores ranged from 1 
(Poor) to 5 (Excellent) and each proposal was typically scored by 3 reviewers 
T p-value is from the Student's T-Test 

The column labeled "Application Cohort CMH" shows the p-values from Cochran-Mantel-Haenszel tests of the 
null hypothesis of no common participation effect across application cohort. 
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To address the potential for bias, the probability of a person responding to the survey both for 
responding and non-responding individuals was estimated as function of baseline characteristics 
that were available for both types of individuals (e.g., proposal score, cohort year, gender), and 
created weighting classes for adjusting the weights of responding individuals to alleviate the bias 
due to non-response. Steps 1-4 described below were taken to accomplish this task. 


Estimating Probability of Response 


Step 1 : Fit Models 

Logistic regression models were fit to estimate the probability of a person responding to the survey. 
The response (dependent) variable is a dummy variable that took the value "1" for responding 
applicants and took the value "0" for non-responding applicants. The explanatory (independent) 
variables are the variables described in Exhibit B.l. Models also included all two-way interaction 
terms of award status/application cohort with the other variables described in Exhibit B.l. 

Several models were fit to identify the set of explanatory variables that have statistically significant 
associations with the dependent variable (p<0.20 criterion) after controlling for other statistically 
significant control variables. This was accomplished by using backwards elimination with forward 
checking. 2 In this method, all of the explanatory variables are entered as predictors in the logistic 


Backwards elimination methods are attractive from the point of view that they are often used and 
familiar. But use of this method with the conventional p<0.05 criterion has been criticized from the point 
of view that the selection criteria tend to favor covariates with strong relationships to the outcome, but 
may omit important confounders (i.e., variables that have a weaker relationship to the outcome, but have 
a strong relationship to the predictor variable of interest). Maldonado and Greenland (1993) evaluated a 
backwards elimination strategy and a change-in-estimate strategy using simulated data from a poisson 
regression model. They found that the p-value based method performed adequately when the alpha 
levels were higher than conventional levels (0.20 or more), and found that the change-in-estimate 
strategy performed adequately when the cut point was set to 10 percent. However, their data, generated 
from a poisson model, and their analysis model, with only a single covariate in addition to the key 
exposure variable, are very different than the models anticipated for the current purpose. 

Budtz-Jorgensen et al. (2001) compared several covariate selection strategies including backwards 
elimination and change-in-estimate. They looked at the backwards elimination strategy with three p-value 
cut-off levels, 0.05, 0.10, and 0.20, and, following the recommendation of Maldonado and Greenland 
(1993) used a 10% criterion for the change-in-estimate method. They found that, although the change-in- 
estimate strategy did an adequate job of identifying confounders and keeping them in the model, it 
sometimes threw out variables that were correlated with the outcome, but were not confounders. 
Therefore, this method threw out variables that, if retained, would have reduced the residual error and 
reduced the standard error of the exposure coefficient (thus increasing the power to detect exposure 
effects - exposure effect is analogous to the key predictor of interest). Although they found that 
backwards elimination with a p<0.05 criterion was un-suited for confounder identification, they found 
that when the p-value criterion was set to p<0.20, backwards elimination strategy resulted in a reduction 
of residual error variance and did not throw out important confounders. They recommended the 
backwards elimination strategy with a p<0.20 criterion over the change-in-estimate strategy. 

Maldonado, G., & Greenland, S. 1993. Simulation study of confounder-selection strategies. American 
Journal of Epidemiology, 138(11), 923-936 
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regression model. The explanatory variable with the largest non-significant value is dropped from 
the subsequent model. This step is repeated until the only explanatory variables that remain in the 
model are those that meet the p<0.20 criterion. In the forwards checking step, each of the 
previously eliminated control variables is checked by adding each one to the model with only the 
significant predictors. In this step, each variable has a chance to get back into the model. The final 
model indicated that the probability of being a responding applicant was related to: final award 
status (nfinalawdfin), gender (cr_gender_nrbias), cohort year applied (nYear_App_group), final 
application score (finalscore), handicap status (cr_hdcap_nrbias), the interaction between final 
award status and gender (nfinalawd*cr_gender_nrbias), and the interaction between cohort year 
and gender (cr_gender*nYear_App_group). The results from the final model are summarized in 
Exhibit B.5. 


Exhibit B.5: Summary of Final Model Results 


Type III Analysis of Effects 


Variable 

DF 

Wald Chi-Square 

Pr > Chisq 

nfinalawdfin 

1 

21.4552 


<.0001** 

cr_gender_nrbias 

1 

2.7613 


0.0966 

nfinalawd*cr_gender_nrbias 

1 

2.3387 


0.1262 

cr_gender_nrbias*nYear_App_group 

2 

4.7379 


0.0936 

FinalScore 

1 

5.4213 


0.0199* 

cr_hdcap_nrbias 

2 

150.0764 

<.0001** 

nYear_App_group 

2 

12.4245 


0.0020** 

The Logistic Procedure 



Analysis of Maximum Likelihood Estimates 





Standard Wald Chi- 


Parameter DF 

Estimate 

Error 

Square 

Pr > ChiSq 

Intercept 1 

-3.5711 

0.5121 

48.6368 

<.0001** 

nfinalawdfin 1 

0.8912 

0.1924 

21.4552 

<.0001** 

cr_gender_nrbias 1 

0.4039 

0.2431 

2.7613 

0.0966 

nfinalawd*cr_gender_nrbias 1 

0.4554 

0.2978 

2.3387 

0.1262 

cr_gender*nYear_App_group_l 1 

-0.3637 

0.3272 

1.2354 

0.2664 

cr_gender*nYear_App_group_2 1 

-0.6737 

0.3101 

4.7191 

0.0298* 

FinalScore 1 

0.2535 

0.1089 

5.4213 

0.0199** 

cr_hdcap_nrbias_0 1 

3.5605 

0.2919 

148.7444 

<.0001** 

cr_hdcap_nrbias_l 1 

2.5692 

0.7631 

11.3345 

0.0008** 

nYear_App_group_l 1 

-0.6808 

0.1932 

12.4164 

0.0004** 

nYear_App_group_2 l 

-0.3950 

0.1924 

4.2152 

0.0401 


Step 2: Use Model Results to Calculate Response Propensities 

In Step 2, parameter estimates obtained from the fitted model were used to calculate the predicted 
probability that an applicant will respond to the survey. The logistic regression model is represented 
as: 


Budtz-Jorgensen, E., Keilding, N., Grandjean, P., Weihe, P., & White, R. 2001. Confounder Identification in 
environmental epidemiology. Assessment of health effects of prenatal mercury exposure. Retrieved from 
http://www.pubhealth.ku.dk/bsa/research-reports/paper_ms.ps 


Abt Associates Inc. 


Appendix B B-6 






Evaluation of International Research Fellowship Program: Final Report 


log( 


1 



71 

where 1 is the probability that applicant i is a responding applicant, and the summation is over the 
k predictor variables in the final model. The predicted probabilities were obtained by solving the 

71 

previous equation for ' , and substituting the parameter estimates from the fitted model in place 
of the parameters. The solution for the predicted probability for applicant i is given by: 


exp( Aj + XA,) 

A 

1 + exp (A + XA,) 

k 

71 

Each applicant's predicted probability of response ( ' ) is called its "response propensity." 
Applicants with similar response propensities have similar characteristics. In particular, they are 
similar on the characteristics that are most related to the probability of response. 

Step 3: Group Applicants with Similar Response Propensities into Weighting Classes 

In this step, applicants with similar response propensities were grouped into weighting classes. The 
weights of responding applicants within a class were inflated so that the responding applicants 
within the class represent the population that both the responding and non-responding applicants 
within the class were originally sampled to represent. Exhibit B.6 shows the distribution of response 
propensities for the applicant sample. 


Exhibit B.6: Distributions of Propensity Scores (All Applicants) 


j Quantile 

Estimate j 

100% Max 

0.9515750 

99% 

0.9485914 

95% 

0.8886858 

90% 

0.8671525 

75% Q3 

0.8143297 

50% Median 

0.6901961 

25% Q1 

0.5703706 

10% 

0.0793978 

5% 

0.0344925 

1% 

0.0254720 

0% Min 

0.0205654 


Weighting classes were formed to ensure that all applicants within a class fell within a narrow range 
of propensity scores. The boundaries for the weighting classes were determined by creating 
approximately equal-interval propensity score groupings. The top and the bottom of each 
propensity interval differed by .10 to .30 points. The resulting five classes corresponded to 
propensities in the ranges of 2-31, 31-60, 61-70, 71-85, and 86-96 percent probability of response. 
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Exhibit B.7 shows the frequency and percent of applicants that fell within each of the five weighting 
classes. 

Exhibit B.7: Number and Percent of Applicants in Each of Five Weighting Classes (Results for All 


Applicants in the Sample) 


Weighting Class 

Frequency 

Percent 

Cumulative 

Frequency 

Cumulative 

Percent 

l)Propensities .0-30 

201 

12.35 

201 

12.35 

2)Propensities .31-. 60 

345 

21.19 

546 

33.54 

3)Propensities .61-. 70 

407 

25.00 

953 

58.54 

4)Propensities .71-. 85 

412 

25.31 

1365 

83.85 

5)Propensities .85-. 96 

263 

16.15 

1628 

100.00 


Step 4. Within Weighting Class, Inflate Weights of Responding Applicants to Sum to 
Population Total 

Within each weighting class, the weights of all applicants (both responders and non-responders) 
were summed. Next, the weights of just the responding applicants were summed. Then, within each 
weighting class, new, adjusted weights of responding applicants were calculated by multiplying the 
initial weights of 1 by a factor equal to the ratio of the sum of the weights of all applicants to the 
sum of the weights of the responding applicants. The adjusted weight for the / th applicants in the y' th 
weighting class is represented symbolically by: 


adi 4, k e responders & nonresponders 

w = w * 

'■ w ij 
i e responders 


z 

rs & > 

z 


w n 


where w u is the initial sampling weight for the /' th applicants in the y' th weighting class, the summation 


in the numerator is over all k applicants in the set of responders and non-responders within 
weighting class j, the summation in the denominator is over all / applicants in the set of responders 
in weighting class j, and there are 7=1,..., 5 weighting classes. The new, adjusted sampling weights 
sum to the population total number of applicants. This result can be written symbolically as: 


Z Z<*=Z Z M ’« 

j i e responders j k e responders & nonresponders 

Exhibit B.8 shows that the weights of the 1628 applicants in the sample sum to the total number of 
applicants in the target population (N=1628). Exhibit B.9 shows the weights summed within each of 
the five weighting classes. The sum shown for the y' th weighting class (j= 1,...,5) corresponds to the 
term: 


IX 

k e responders & nonresponders 

For example, the applicant weights of the 201 applicants in the first weighting class sum to: 

Z w » = 201 - 

k e responders & nonresponders 
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Exhibit B.8: Size of Target Population (Sum of Initial Weights for All Applicants in the Sample) 


I Weight Variable 

N 

Sum 

Minimum 

Maximum I 

Appl_wgt 

1628 

1628 

1.00 

1.00 


Appl_wgt=the initial weight for all applicants in the sample (sums to the population total number of applicants) 


Exhibit B.9: Size of Target Population Within Each Weighting Class (Sum of Weights for All 
Applicants in the Sample) 


Weighting Class 

N 

Sum 

Minimum 

Maximum 1 

l)Propensities 

.0-.30 

201 

201 

1.00 

1.00 

2)Propensities 

.31-. 60 

345 

345 

1.00 

1.00 

3)Propensities 

.61-. 70 

407 

407 

1.00 

1.00 

4)Propensities 

.71-. 85 

412 

412 

1.00 

1.00 

5)Propensities 

.85-. 96 

263 

263 

1.00 

1.00 


Appl_wgt=initial weight for the applicant sample (sums to population total number of applicants within class) 


The sum of the initial weights of the 1,039 responding applicants is shown in Exhibit B.10. These are 
the "initial weights" because they are the sampling weights prior to adjustment for non-response. 
The weights of the 1,039 responding applicants sum to a number that is smaller than the size of the 
target population. Exhibit A. 11 shows the weights summed within each of the five weighting classes. 
The sum shown for the / h weighting class (/= 1,...,5) corresponds to the term: 

!></■• 

i e responders 

For example, the initial applicant weights of the n responding applicants in the first weighting class 
sum to: 


=12 

i e responders 


Exhibit B.10: Initial (Unadjusted) Weights of Responding Applicants (Sum of Initial Weights for 
All 1,039 Responding Applicants) 


Variable 

N 

Sum 

Minimum 

Maximum | 

Appl_wgt 

1,039 

1,039 

1.00 

1.00 


Appl_wgt=the initial weight for the applicant sample 
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Exhibit B.ll: Initial (Unadjusted) Weights of Responding Applicants by Weighting Class (Sum of 


Initial Weights for All Responding Applicants) 


| Weighting Class 

N 

Sum 

Minimum 

Maximum 1 

l)Propensities 

.0-.30 

12 

12 

1.00 

1.00 

2)Propensities 

.31-. 60 

181 

181 

1.00 

1.00 

3)Propensities 

.61-. 70 

287 

287 

1.00 

1.00 

4)Propensities 

.71-. 85 

324 

324 

1.00 

1.00 

5)Propensities 

.85-. 96 

235 

235 

1.00 

1.00 


Appl_wgt=initial weight for the applicant sample (sums to population total number of applicants within class) 


The inflation factors for each of the five weighting classes are shown in Exhibit A. 12. The inflation 
factors correspond to the term: 


I >*/ 

k e responders & nonresponders 

Z W u 

i e responders 

For example, the inflation factor for the first weighting class is: 


12 

Exhibit B.12: Inflation Factors Within Weighting Classes 


Weighting Class 

All Applicants 

l)Propensities .0-.30 

16.75 

2)Propensities .31-. 60 

1.9060773481 

3)Propensities .61-. 70 

1.4181184669 

4)Propensities .71-. 85 

1.2716049383 

5)Propensities .85-. 96 

1.1191489362 


Exhibits B.13 and B.14 show the sums of the non-response adjusted weights for the 1039 
responding applicants, overall and by weighting class. The adjusted weights sum to the size of the 
target population. The numbers shown in Exhibits B.13 and B.14 correspond to the term: 

I K 

j i e responders 

For example, the non-response adjusted applicant weights of the 1039 responder applicants sum to 

X £<= 1628 , 

j i e responders 

and, for example, the non-response adjusted applicant weights of the 1039 responding applicants in 
the bottom weighting class sum to 


Z< = 263. 

i e responders 
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Exhibit B.13: Sum of Nonresponse Adjusted Weights for 1039 Responding Applicants 

Variable 

N 

Sum 

Minimum 

Maximum 

appl_wgt_adj 

1039 

1628 

1.1191489 

16.7500000 

appl_wgt_adj=the adjusted weight for the applicant sample 

Exhibit B.14: Sum of Nonresponse Adjusted Weights for 1039 Responding Applicants by 

Weighting Class 





Weighting Class 

N 

Sum 

Minimum 

Maximum 

l)Propensities .0-.30 

201 

201 

16.75 

16.75 

2)Propensities .31-. 60 

345 

345 

1.91 

1.91 

3)Propensities .61-. 70 

407 

407 

1.42 

1.42 

4)Propensities .71-. 85 

412 

412 

1.27 

1.27 

5)Propensities .85-. 96 

263 

263 

1.12 

1.12 


Appl_wft_adj=the adjusted weight for the applicant sample 


Item Non-Response 

Item non-response refers to one or more specific uncompleted items on an otherwise 
completed/returned questionnaire. Since the amount of missing data on an individual item was 
modest (<5 percent across all returned surveys), descriptive statistics were calculated on only the 
non-missing items, which is equivalent to an assumption that missing data on an item are missing 
completely at random. The amount of missing data for each item is presented in all tables/figures 
included in reports. 

Where necessary for the impact analyses, distinct approaches to imputing values were taken 
depending on whether data were missing for an item used to construct a covariate or predictor 
variable, or an outcome variable. For impact analyses where missing data on covariate or predictor 
variables require imputation to prevent having to omit those respondents from the analysis, a 
"dummy-variable" method was used. This method entailed (i) creating a dummy variable that equals 
“ 1 ” if the value of the variable is missing and "0" otherwise, (ii) adding the dummy variable to the 
impact model as a covariate, and (iii) replacing the missing value of the original variable with 
predicted values from a logistic or linear regression model (see Appendix C for more details). 

If the missing data occurred in an item used to construct an outcome— that is, one of the primary 
outcomes of interest (for example, the post-fellowship number of publications produced with a 
foreign co-author)— no imputation was conducted. 
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Appendix C: Detailed Description of Impact Analysis 


This appendix describes the methodology used to estimate the effects of the IRFP award on its 
recipients. First, key features of the quasi-experimental design are reviewed in order to provide 
context for a general discussion of the use of propensity score analysis (PSA) in mitigating selection 
bias. Next, the steps used to match awardees and non-awardees are presented in detail, including 
the estimated logistic model and the resulting distribution of awardees and non-awardees in the 
propensity strata. Finally, specifications of the impact models and sensitivity analyses conducted are 
presented. 


Propensity Score Matching 


One of the main purposes of this evaluation was to estimate the effect of IFRP on its participants. If 
a program brings about changes in its participants, then these individuals should have different 
outcomes, post-participation, than they would have had in the absence of program participation. 

Questions about the impact of a program seek to determine whether any observed differences 
between participants and non-participants can be attributed to the program itself rather than to 
other, non-program related causes. One potential source of other causes for differences between 
the two groups is pre-existing characteristics that could affect both selection into the program and 
post-program differences, often called "selection bias." Propensity score matching (PSM) was used 
to mitigate selection bias for this evaluation. 

Propensity score matching is one type of propensity score analysis (PSA) wherein participants in a 
program are matched to non-participants on the basis of their "participation propensity score." This 
technique uses pre-treatment characteristics to determine the probability (i.e., the propensity 
score) that applicants would be selected for the treatment (namely, an IRFP award) based on known 
pre-existing characteristics. After assigning a propensity score to each individual, applicants are 
placed into blocks (or matching strata) such that the actual IRFP awardees and the unfunded 
applicants within each block have approximately equal predicted propensity to be in the treated 
group. The quasi-experimental estimates of the impact of the program can then be obtained by 
comparing the outcomes of awardees and non-awardees within each propensity block and 
aggregating the differences across the blocks. This is accomplished by including terms for the 
propensity blocks in the models used for analysis. 

PSM was performed via the following four steps: 

Step 1 : Identify the pre-treatment characteristics that will be used in the propensity score 
model to match fellows and unfunded applicants. 

The pre-treatment characteristics to be used in the propensity score model to match fellows and 
unfunded applicants were identified. These characteristics included variables that both predicted 
receiving the fellowship and might also affect the outcomes of interest. They were taken from NSF 
extant data and applicant survey data. Exhibit C.l shows the variables used in the propensity score 
model. 
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Some of the characteristics to be used in the PSA had missing data. If these missing data had been 
ignored, records that had missing data in the PSA matching would have been lost. A recommended 
approach for addressing this issue is to do a simple single imputation of the missing covariates and 
include missing data indicators in the propensity score model. This method essentially matches both 
on the observed values and on the missing data patterns. * 3 Although it cannot balance the missing 
data values themselves this method will yield balance on the covariates and the missing data 
patterns. 4 

Single imputation substitutes a missing value with a definite value following an established 
procedure. Predicted values from a logistic or linear regression model were used to impute a 
definite value for missing values in this study. The outcomes for these models were the PSA 
variables that had missing data and the predictors were all PSA variables with non-missing data. 
Imputation was done first for the variable with the least amount of missing. This variable was then 
added to the right hand side of the model for the next variable with the least amount of missing. 

This process continued through to the variable with the most number of cases missing with all of the 
other PSA variables. For binary variables a imputation was done using randomly generated a 0 or 1 
using a binomial distribution with p=predicted probability from the model discussed above. For 
continuous variable the predicted values generated from the model was used as the imputed value. 
Imputation flags were created to indicate if an observation was imputed or not to include in the PSA 
model. Exhibit C.2 shows the distribution of each variable overall and by award status prior and 
post-imputation. 


Stuart, E.A. 2010. Matching methods for casual inference: A review and a look forward. Statistical Science, 

25(1), 1-21. 

4 Rosenbaum, P.R., & Rubin, D.B. 1984. Reducing bias in observational studies using subclassification on the 
propensity score. Journal of the American Statistical Association, 79, 516 -524. 
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Exhibit C.l: Pre-Award Data Used to Construct Comparable Groups of IRFP Fellows and Unfunded Applicants for Impact Models 


Pre-Award Characteristic 

Reason for Inclusion in Propensity 
Score Model 1 

Data Source(s) 

Type 

Definition 

Mean proposal score 

Proposal score indicates quality of 
application 

NSF Extant Data 

Continuous 

Average score across reviews (1-5) 

Cohort year 

Control for cohort differences 

NSF Extant Data 

Dichotomous 

1992 (0/1) 


1993 (0/1) 
1994 (0/1) 
1995 (0/1) 
1996 (0/1) 
1997 (0/1) 
1998 (0/1) 
1999 (0/1) 
2000 (0/1) 
2001 (0/1) 
2002 (0/1) 
2003 (0/1) 
2004 (0/1) 
2005 (0/1) 
2006 (0/1) 
2007 (0/1) 
2008 (0/1) 
2009 (0/1) 


Geographic density 

Desire for geographic balance in 
portfolio 3 

NSF Extant Data 

Dichotomous 

North America (0/1) 
South/Central America (0/1) 
Europe (0/1) 

Africa /Middle East (0/1) 
South Pacific (0/1) 

East Asia (0/1) 

Gender 

Preference given for females (women 
encouraged to apply) 3 

IFRP Applicant Survey: 
G1 

Dichotomous 

l=Female 

0=Male 

Under-represented minority 
status 

Preference given for under- 
represented minority status (members 
of these groups were encouraged to 
apply) 3 

IFRP Applicant Survey: 
G2, G3 

Dichotomous 

l=Other race(s)/ethnicity 0=Asian 
Only or White Only 
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Exhibit C.l: Pre-Award Data Used to Construct Comparable Groups of IRFP Fellows and Unfunded Applicants for Impact Models 

Pre-Award Characteristic 

Reason for Inclusion in Propensity 
Score Model 1 

Data Source(s) 

Type 

Definition 

Citizenship status 

US citizenship (birth, naturalized) or 
permanent residency required 3 

IFRP Applicant Survey: 
G4, G4a 

Dichotomous 

1=US Citizen/Permanent Resident 
0=Non-US Citizen or Permanent 
Resident 

Disability status 

Preference given to disabled applicant 
(disabled applicants encouraged to 
apply) 3 

IFRP Applicant Survey: 
G5 

Dichotomous 

l=Disabled 

0=Not-Disabled 

STEM discipline 

Desire for disciplinary balance in 
program portfolio 3 

IFRP Applicant Survey: 
A5 

Dichotomous 

Biological, agricultural, or 
environmental life sciences (1/0) 
Physical and related sciences 
(includes Chemistry except 
biochem; earth, atmospheric, 
ocean sciences; physics) (1/0) 
Computer and information sciences 
& Mathematics and statistics (1/0) 
Psychology & social sciences 
&related science (1/0) 

Engineering (1/0) 

Health (non-STEM) 1/0) 

Had tenure-track position 

Priority given to applicants not in a 
tenure-track position 3 

IFRP Applicant Survey: 
Cl 

Dichotomous 

l=Not on tenure track 
0=On tenure track 

Highest degree held 

Need to have a PhD at time of award 3 

IFRP Applicant Survey: 
C2a 

Dichotomous 

l=Doctoral degree (PhD, EdD, MD, 
joint MD/PhD, JD, PsyD, ScD) 
0=Other Degree 

Degree from non-US 
institution 

Lower priority given to applicants with 
a PHD from a foreign institution 3 

IFRP Applicant Survey: 
C2b 

Dichotomous 

l=No degree from a non-US 
institution 

0=Degree from a non-US institution 

Study-abroad as 
undergraduate 

Preference given to those with no 
prior international experience 3 

IFRP Applicant Survey: 
C3a 

Dichotomous 

l=No prior experience studying 
abroad 

Study-abroad as grad 
student 

Preference given to those with no 
prior international experience 
(applicant must justify award) 3 

IFRP Applicant Survey: 
C3b 

Dichotomous 

0=Prior experience studying abroad 

Prior international 
residential 

Preference given to those with no 
prior international experience 3 

IFRP Applicant Survey: 
C4 

Dichotomous 

l=No prior experience living abroad 
0=Prior experience living abroad 
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Exhibit C.l: Pre-Award Data Used to Construct Comparable Groups of IRFP Fellows and Unfunded Applicants for Impact Models 

Pre-Award Characteristic 

Reason for Inclusion in Propensity 
Score Model 1 

Data Source(s) 

Type 

Definition 

Prior international 
collaboration 

Likely to be beneficial to applicant 

IFRP Applicant Survey: 
C4, C5b 

Dichotomous 

l=Prior international collaboration 
0=No prior international 
collaboration 

Already at host institution 

Unfavorable for IRFP applicant 
(applicant must justify award) a 

IFRP Applicant Survey: 
C5a 

Dichotomous 

l=Not working at a foreign 
institution 

0=Working at a foreign institution 

Prior international exposure 

Prior exposure to foreign colleagues or 
former program fellow favorable to 
applicant 

IFRP Applicant Survey: 
C4 

Dichotomous 

l=Prior exposure to foreign 
colleagues or former program 
fellow 

0=No prior exposure to foreign 
colleagues or former program 
fellow 

Link between US, host 
institutions 

Likely to be beneficial to applicant 

IFRP Applicant Survey: 
C6 

Dichotomous 

l=Link between US, host 
institutions 

0=No Link between US, host 
institutions 

Total pre-award 
publications 

Prior record of achievement 
favorable 3 

IFRP Applicant Survey: 
Cl 

Continuous 

Total pre-award publication 

% publications w/foreign 
collaborator 

Likely to be beneficial to applicant 

IFRP Applicant Survey: 
C7 

Continuous 

Percent of publications with foreign 
collaborator 

National post-collegiate 
fellowship 

Prior record of achievement favorable 

a 

C8 

Dichotomous 

l=Received a national post- 
collegiate fellowship 
0=Did not received any national 
post-collegiate fellowship 

a These criteria were based on 

information provided in one or more of the following NSF solicitations: 00-141, 01-135, 

02-149, 05-599, and 06-582. 
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Exhibit C.2: Distribution of Pre-Award Data Used To Construct Comparable Groups of IRFP Fellows and Unfunded Applicants for Impact 
Models 


Prior to Imputation 

Post-Imputation 

Pre-Award 

Characteristic Variable N 

N Mean/ 

miss Min Max Percent 

N Mean/ 

N miss Min Max Percent 


Among All Applicants 


Mean Proposal 
Score 

finalscore 

1039 

0 

1.33 

5 

4.1 

1039 

0 

1.33 

5 

4.1 

Cohort Year 

CohortYr 

1039 

0 

1992 

2009 

2002.7 

1039 

0 

1992 

2009 

2002.7 

Geographic Density 

NrthAmerica_PSM 

1010 

29 

0 

1 

9.6% 

1039 

0 

0 

1 

9.3% 


SthCenAmerica_PSM 

1010 

29 

0 

1 

8.8% 

1039 

0 

0 

1 

8.6% 


Europe_PSM 

1010 

29 

0 

1 

59.5% 

1039 

0 

0 

1 

60.3% 


AfricaME_PSM 

1010 

29 

0 

1 

5.8% 

1039 

0 

0 

1 

6.1% 


SthPac_PSM 

1010 

29 

0 

1 

10.8% 

1039 

0 

0 

1 

10.5% 


EAsia_PSM 

1010 

29 

0 

1 

5.4% 

1039 

0 

0 

1 

5.3% 

Gender 

Female_PSM 

1039 

0 

0 

1 

59.2% 

1039 

0 

0 

1 

59.2% 

Under- 
Represented 
Minority Status 

crurmPSM 

1029 

10 

0 

1 

7.0% 

1039 

0 

0 

1 

7.0% 

Citizenship Status 

Citizen_PSM 

1034 

5 

0 

1 

99.8% 

1039 

0 

0 

1 

99.8% 

Disability status 

hdcap_PSM 

1025 

14 

0 

1 

0.4% 

1039 

0 

0 

1 

0.4% 

STEM Discipline 

Bioagenv_PSm 

1039 

0 

0 

1 

47.6% 

1039 

0 

0 

1 

47.6% 


CompMathStat_PSM 

1039 

0 

0 

1 

6.0% 

1039 

0 

0 

1 

6.0% 


Physical_PSM 

1039 

0 

0 

1 

27.3% 

1039 

0 

0 

1 

27.3% 


Socal_PSM 

1039 

0 

0 

1 

6.8% 

1039 

0 

0 

1 

6.8% 


Engineering_PSM 

1039 

0 

0 

1 

12.0% 

1039 

0 

0 

1 

12.0% 


Health_PSM 

1039 

0 

0 

1 

0.2% 

1039 

0 

0 

1 

0.2% 

Tenure Track 
Position 

Notenuretrack_PSM 

1037 

2 

0 

1 

94.7% 

1039 

0 

0 

1 

94.7% 

Graduate Degree 
Program 

PhD_PSM 

1039 

0 

0 

1 

94.4% 

1039 

0 

0 

1 

94.4% 

Degree from 
Foreign Institution 

Noforeigndegree_PSM 

1035 

4 

0 

1 

92.0% 

1039 

0 

0 

1 

92.0% 

Study Abroad as an 

NoStudAbr_PSM 

1034 

5 

0 

1 

43.2% 

1039 

0 

0 

1 

43.0% 


undergraduate or 
graduate 
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Exhibit C.2: Distribution of Pre-Award Data Used To Construct Comparable Groups of IRFP Fellows and Unfunded Applicants for Impact 
Models 


Prior to Imputation Post-Imputation 


Pre-Award 



N 



Mean/ 


N 



Mean/ 

Characteristic 

Variable 

N 

miss 

Min 

Max 

Percent 

N 

miss 

Min 

Max 

Percent 

Prior visit to host 
location 

Nopriorres_PSM 

1039 

0 

0 

1 

55.2% 

1039 

0 

0 

1 

55.2% 

Prior international 
collaboration 

priorcollab_PSM 

1035 

4 

0 

1 

54.4% 

1039 

0 

0 

1 

54.3% 

Located at Host 
University 

Notathost_PSM 

1034 

5 

0 

1 

86.4% 

1039 

0 

0 

1 

86.4% 

Prior international 

priorexpose_PSM 

1035 

4 

0 

1 

72.7% 

1039 

0 

0 

1 

72.7% 

exposure 












Link between host, 
US institution 

linkhost_PSM 

1030 

9 

0 

1 

34.0% 

1039 

0 

0 

1 

33.8% 

Total Pre-Award 
publications 

AllWorks 

990 

49 

0 

80 

9.3 

1039 

0 

0 

80 

9.3 

Percent of 
publications with a 
foreign 
collaborator 

percentworks 

983 

56 

0 

100 

16.1 

1039 

0 

0 

100 

16.0 

National post- 

collegiate 

fellowship 

fellow_PSM 

1026 

13 

0 

1 

45.5% 

1039 

0 

0 

1 

45.1% 

Among Awardees Only 











Mean Proposal 
Score 

finalscore 

457 

0 

3 

5 

4.5 

457 

0 

3 

5 

4.5 

Cohort Year 

CohortYr 

457 

0 

1992 

2009 

2001.9 

457 

0 

1992 

2009 

2001.9 

Geographic Density 

N rth America_PSM 

457 

0 

0 

1 

9.2% 

457 

0 

0 

1 

9.2% 


SthCenAmerica_PSM 

457 

0 

0 

1 

10.3% 

457 

0 

0 

1 

10.3% 


Europe_PSM 

457 

0 

0 

1 

58.6% 

457 

0 

0 

1 

58.6% 


AfricaME_PSM 

457 

0 

0 

1 

6.3% 

457 

0 

0 

1 

6.3% 


SthPac_PSM 

457 

0 

0 

1 

10.3% 

457 

0 

0 

1 

10.3% 


EAsia_PSM 

457 

0 

0 

1 

5.3% 

457 

0 

0 

1 

5.3% 

Gender 

Female_PSM 

457 

0 

0 

1 

59.1% 

457 

0 

0 

1 

59.1% 
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Exhibit C.2: Distribution of Pre-Award Data Used To Construct Comparable Groups of IRFP Fellows and Unfunded Applicants for Impact 
Models 






Prior to Imputation 




Post-Imputation 


Pre-Award 



N 



Mean/ 


N 



Mean/ 

Characteristic 

Variable 

N 

miss Min 

Max 

Percent 

N 

miss 

Min 

Max 

Percent 

Under-Represented 
Minority Status 

cr_urm_PSM 

452 

5 

0 

1 

7.7% 

457 

0 

0 

1 

7.9% 

Citizenship Status 

Citizen_PSM 

455 

2 

0 

1 

99.6% 

457 

0 

0 

1 

99.6% 

Disability status 

hdcap_PSM 

450 

7 

0 

1 

0.2% 

457 

0 

0 

1 

0.2% 

STEM Discipline 

Bioagenv_PSm 

457 

0 

0 

1 

43.3% 

457 

0 

0 

1 

43.3% 


CompMathStat_PSM 

457 

0 

0 

1 

6.3% 

457 

0 

0 

1 

6.3% 


Physical_PSM 

457 

0 

0 

1 

29.5% 

457 

0 

0 

1 

29.5% 


Socal_PSM 

457 

0 

0 

1 

8.8% 

457 

0 

0 

1 

8.8% 


Engineering_PSM 

457 

0 

0 

1 

11.8% 

457 

0 

0 

1 

11.8% 


Health_PSM 

457 

0 

0 

1 

0.2% 

457 

0 

0 

1 

0.2% 

Tenure Track 
Position 

Notenuretrack_PSM 

457 

0 

0 

1 

95.0% 

457 

0 

0 

1 

95.0% 

Graduate Degree 
Program 

PhD_PSM 

457 

0 

0 

1 

97.8% 

457 

0 

0 

1 

97.8% 

Degree from 
Foreign Institution 

Noforeigndegree_PSM 

457 

0 

0 

1 

92.1% 

457 

0 

0 

1 

92.1% 

Study Abroad as an 
undergraduate or 
graduate 

NoStudAbr_PSM 

456 

1 

0 

1 

45.4% 

457 

0 

0 

1 

45.3% 

Prior visit to host 
location 

Nopriorres_PSM 

457 

0 

0 

1 

52.7% 

457 

0 

0 

1 

52.7% 

Prior international 
collaboration 

priorcollab_PSM 

455 

2 

0 

1 

56.9% 

457 

0 

0 

1 

56.9% 

Located at Host 
University 

Notathost_PSM 

456 

1 

0 

1 

90.1% 

457 

0 

0 

1 

90.2% 

Prior international 

priorexpose_PSM 

455 

2 

0 

1 

73.4% 

457 

0 

0 

1 

73.5% 

exposure 












Link between host, 
US institution 

linkhost_PSM 

454 

3 

0 

1 

34.6% 

457 

0 

0 

1 

34.4% 

Total Pre-Award 
publications 

totalworks 

441 

16 

0 

55 

9.1 

457 

0 

0 

55 

9.1 
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Exhibit C.2: Distribution of Pre-Award Data Used To Construct Comparable Groups of IRFP Fellows and Unfunded Applicants for Impact 


Models 





Prior to Imputation 




Post-Imputation 


Pre-Award 



N 



Mean/ 


N 



Mean/ 

Characteristic 

Variable 

N 

miss Min 

Max 

Percent 

N 

miss 

Min 

Max 

Percent 

Percent of 
publications with a 
foreign 
collaborator 

percentworks 

437 

20 

0 

100 

16.5 

457 

0 

0 

100 

16.3 

National post- 

collegiate 

fellowship 

fellow_PSM 

454 

3 

0 

1 

49.8% 

457 

0 

0 

1 

49.2% 

Among Declines Only 

Mean Proposal 
Score 

finalscore 

582 

0 

1.33 

5 

3.7 

582 

0 

1.33 

5 

3.7 

Cohort Year 

CohortYr 

582 

0 

1992 

2009 

2003.4 

582 

0 

1992 

2009 

2003.4 

Geographic Density 

NrthAmerica_PSM 

553 

29 

0 

1 

9.9% 

582 

0 

0 

1 

9.5% 


SthCenAmerica_PSM 

553 

29 

0 

1 

7.6% 

582 

0 

0 

1 

7.2% 


Europe_PSM 

553 

29 

0 

1 

60.2% 

582 

0 

0 

1 

61.5% 


Africa ME_PSM 

553 

29 

0 

1 

5.4% 

582 

0 

0 

1 

5.8% 


SthPac_PSM 

553 

29 

0 

1 

11.2% 

582 

0 

0 

1 

10.7% 


EAsia_PSM 

553 

29 

0 

1 

5.6% 

582 

0 

0 

1 

5.3% 

Gender 

Female_PSM 

582 

0 

0 

1 

59.3% 

582 

0 

0 

1 

59.3% 

Under-Represented 
Minority Status 

cr_urm_PSM 

577 

5 

0 

1 

6.4% 

582 

0 

0 

1 

6.4% 

Citizenship Status 

Citizen_PSM 

579 

3 

0 

1 

100.0% 

582 

0 

0 

1 

100.0% 

Disability status 

hdcap_PSM 

575 

7 

0 

1 

0.5% 

582 

0 

0 

1 

0.5% 

STEM Discipline 

Bioagenv_PSm 

582 

0 

0 

1 

51.0% 

582 

0 

0 

1 

51.0% 


CompMathStat_PSM 

582 

0 

0 

1 

5.7% 

582 

0 

0 

1 

5.7% 


Physical_PSM 

582 

0 

0 

1 

25.6% 

582 

0 

0 

1 

25.6% 


Socal_PSM 

582 

0 

0 

1 

5.3% 

582 

0 

0 

1 

5.3% 


Engineering_PSM 

582 

0 

0 

1 

12.2% 

582 

0 

0 

1 

12.2% 


Health_PSM 

582 

0 

0 

1 

0.2% 

582 

0 

0 

1 

0.2% 

Tenure Track 
Position 

Notenuretrack_PSM 

580 

2 

0 

1 

94.5% 

582 

0 

0 

1 

94.5% 
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Exhibit C.2: Distribution of Pre-Award Data Used To Construct Comparable Groups of IRFP Fellows and Unfunded Applicants for Impact 
Models 






Prior to Imputation 




Post-Imputation 


Pre-Award 

Characteristic 

Variable 

N 

N 

miss 

: Min 

Max 

Mean/ 

Percent 

N 

N 

miss 

Min 

Max 

Mean/ 

Percent 

Graduate Degree 
Program 

PhD_PSM 

582 

0 

0 

1 

91.8% 

582 

0 

0 

1 

91.8% 

Degree from 
Foreign Istitution 

Noforeigndegree_PSM 

578 

4 

0 

1 

91.9% 

582 

0 

0 

1 

91.9% 

Study Abroad as an 
undergraduate or 
graduate 

NoStudAbr_PSM 

578 

4 

0 

1 

41.5% 

582 

0 

0 

1 

41.2% 

Prior visit to host 
location 

Nopriorres_PSM 

582 

0 

0 

1 

57.2% 

582 

0 

0 

1 

57.2% 

Prior international 
collaboration 

priorcollab_PSM 

580 

2 

0 

1 

52.4% 

582 

0 

0 

1 

52.2% 

Located at Host 
University 

Notathost_PSM 

578 

4 

0 

1 

83.4% 

582 

0 

0 

1 

83.5% 

Prior international 
exposure 

priorexpose_PSM 

580 

2 

0 

1 

72.1% 

582 

0 

0 

1 

72.0% 

Link between host, 
US institution 

linkhost_PSM 

576 

6 

0 

1 

33.5% 

582 

0 

0 

1 

33.3% 

Total Pre-Award 
publications 

totalworks 

549 

33 

0 

80 

9.5 

582 

0 

0 

80 

9.5 

Percent of 
publications with a 
foreign 
collaborator 

percentworks 

546 

36 

0 

100 

15.9 

582 

0 

0 

100 

15.8 

National post- 

collegiate 

fellowship 

fellow_PSM 

572 

10 

0 

1 

42.1% 

582 

0 

0 

1 

41.9% 
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Step 2: Fit a logistic model that predicts the probability of being awarded a fellowship based 
on pre-treatment characteristics. 

The participation propensity score for each individual was estimated using a logistic model with the 
identified pre-treatment characteristics (see exhibit C.3) as the independent variables and receipt of 
the fellowship (dummy-coded as 0 or 1) as the dependent variable. In general, variables were not 
excluded from the logistic model merely because of a lack of significance, i.e., variables were 
included regardless of whether they predicted treatment. Collinearity was accepted among the 
predictors because the model was not intended to predict anything outside the sample space. 
Exhibit C.3 displays the resulting logistic model. The coefficients from this model were used to 
estimate the propensity score for each individual, which represents the probability of receiving a 
fellowship. 


Exhibit C.3: Propensity Score Logistic Model 


Variable 

Coefficient 

Standard Error 

Prob > | z | 

FinalScore 

4.0709 

0.2668 

0.000 

CohortYr 

-0.0854 

0.0192 

0.000 

SthCenAmerica_PSM 

-1.7672 

0.6802 

0.009 

Europe_PSM 

0.1085 

0.3065 

0.723 

AfricaME_PSM 

-0.9404 

0.4809 

0.051 

SthPac_PSM 

-0.2880 

0.3990 

0.470 

EAsia_PSM 

-1.1390 

0.5188 

0.028 

Female_PSM 

0.2455 

0.1835 

0.181 

cr_urm_PSM 

-1.3068 

0.3926 

0.001 

cr_urm_PSM_imp_flg 

0.4782 

1.1408 

0.675 

hdcap_PSM 

0.7882 

1.2939 

0.542 

hdcap_PSM_imp_flg 

-0.3034 

0.8680 

0.727 

Bioagenv_psm 

1.2655 

3.0182 

0.675 

CompMathStat_PSM 

1.0035 

3.0395 

0.741 

Physical_PSM 

0.8655 

3.0224 

0.775 

SocalPSM 

0.3697 

3.0330 

0.903 

Engineering_PSM 

0.9304 

3.0326 

0.759 

Notenuretrack_PSM 

-1.0094 

0.4413 

0.022 

PhD_PSM 

-1.3322 

0.4660 

0.004 

Noforeigndegree_PSM 

0.2319 

0.3410 

0.496 

NoStudAbr_PSM 

-0.0668 

0.2091 

0.749 

Nopriorres_PSM 

0.5391 

0.3705 

0.146 

priorcollab_PSM 

-0.2036 

0.2172 

0.349 

Notathost_PSM 

-1.1177 

0.2800 

0.000 

priorexpose_PSM 

0.5182 

0.3513 

0.140 

priorexpose_imp_flg 

-1.4254 

1.7302 

0.410 

linkhostPSM 

-0.0050 

0.1927 

0.980 

linkhost_PSM_imp_flg 

1.1409 

1.5081 

0.449 

allworks 

-0.0120 

0.0109 

0.270 

allworks_imp_flg 

-0.8153 

0.9193 

0.375 

Percentworks 

0.0023 

0.0039 

0.561 

fellow_psm 

0.1804 

0.1880 

0.337 

fellowJmp_flg 

1.7979 

0.9577 

0.060 

nopriorexpose 

-0.5481 

0.4193 

0.191 

urm_link_flg 

-2.1939 

2.5238 

0.385 

Sthpriorex 

0.6974 

0.7249 

0.336 
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Exhibit C.3: Propensity Score Logistic Model 


§ Variable 

Coefficient 

Standard Error 

Prob > | z | 

_cons 

158.8441 

47.4621 

0.001 

Number of observations=1039 

Pseudo R 2= 0.4247 



LR chi 2 (36)=605.29 

Log-likelihood=-409. 99761 




Prob > chi 2 =0.0000 

NOTES: In estimating the propensity score through a probability model, the choice of which interaction term to include is 
determined solely by the need to condition fully on the observable characteristics that make up the assignment 
mechanism. When covariates are not balanced within a particular stratum, the solution adopted is to divide the stratum 
into finer strata and test again for no difference in the distribution of covariates within the finer strata. If however, some 
covariates still remain unbalanced, the score may be poorly estimated which suggests that additional terms (interaction 
or higher-order terms) of the unbalanced covariates should be added to the logistic specification to control better for 
these characteristics. This procedure is repeated until the covariates are balanced. 5 

Step 3: Use the estimated propensity scores to create matched sets of fellows and unfunded 
applicants. 

The estimated propensity scores were used to create matched sets of fellows and unfunded 
applicants. Propensity scores can be utilized in a number of ways, including matching, stratification, 
weighting, and regression adjustment. * * 6 Stratification (also called interval matching) was the primary 
method of matching. This method was chosen because it allows for the inclusion of the largest 
number of cases and does not impose a functional form (e.g., linear) on the relationship between 
propensity to participate and treatment effect. Once the propensity scores were assigned, the 
region of common support was defined as [.00258812, .9961853], Individuals were identified and 
excluded from the analyses if they were outside of the "common support" group— the range of 
common scores across fellows and unfunded applicants. Enforcing the common support is 
important to ensure the similarity of the matched non-awardees to awardees. 7 Zero awardees and 
32 non-awardees were dropped because their propensity score was outside of this range. 

The remaining individuals were divided into strata within which the awardee and non-awardee 
groups had statistically the same 8 mean propensity score. Within each of these strata tests were 
conducted to identify significant differences between the treatment and comparison group on the 
independent variables. The model was adjusted until all such differences were removed. 9 

The treatment and comparison groups were divided into seven strata based on their propensity 
scores. Research has indicated that at least five strata are generally sufficient for removing 90 


See Deheija, R.H., & Wahba, S. 2002. Propensity score-matching methods for non-experimental casual 

studies. The Review of Economics and Statistics, 84(1), 161. 

6 Hirano, K., Imbens, G.W., & Ridder, G. 2003. Efficient estimation of average treatment effects using the 
estimated propensity score. Econometrica, 71(4), 1161-1189; Morgan, S.L., & Harding, D. J. 2006. 
Matching estimators of causal effects: Prospects and pitfalls in theory and practice. Sociological Methods 
& Research, 35(1), 3-60; and Abadie, A., & Imbens, G.W. 2009. Matching on the estimated propensity 
score (NBER Working Paper 15301). 

7 Rosenbaum & Rubin, 1984; Caliendo, M., & Kopeinig, S. 2007. Some Practical guidance for the 
implementation of propensity score matching. Journal of Economic Surveys, 22(1), 31-72. 

8 Based on level p<0.01 t-tests. 

9 The Intercooled STATA10 pscore process was used to obtain propensity score balance. 
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percent or more of the bias due to the covariates. 10 Exhibit C.4 shows the distribution of the 
treatment and comparison group members by propensity score strata. Exhibit C.5 shows a 
histogram of propensity scores for the treatment and comparison groups. 


Exhibit C.4: Number of Awardee and Non-Awardee in each Propensity Score Block 


| Inferior of Block of Pscore 

Non-Awardee 

Awardee 

Total 

0.0025881 

175 

5 

180 

0.05 

61 

10 

71 

0.1 

76 

14 

90 

0.2 

98 

39 

137 

0.4 

82 

56 

138 

0.6 

45 

101 

146 

0.8 

13 

232 

245 

Total 

550 

457 

1,007 

Number outside the region of common support 

32 

0 

32 


Exhibit C.5: Histograms of Propensity Scores 



Awardees 


Rosenbaum & Rubin, 1984. 
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Step 4: Test whether there are any differences between the awardees and non-awardees 
within each propensity score strata. 

This analysis was conducted in two ways. First, t-tests were used to determine if there were any 
difference for awardees and non-awardees within each stratum for each pre-treatment 
characteristic. 11 As this test is sensitive to sample size (i.e., they tend fail to detect sizable 
differences in small samples, but detect slight differences in larger samples), this analysis was 
supplemented using standardized differences. 12 The standardized difference of a matching 
characteristic between awardees and non-awardees in a given propensity score stratum is 
calculated using the following formula: 


X denotes the variable of interest; 

S denotes the stratum; 

T denotes the treatment group, and Cdenotes the comparison group; 

X T s and X c s denote the treatment and comparison group mean of X in stratum S ; and 

<7 2 x,t and a 2 x.c denote the overall variance of X in the treatment and comparison group, 
respectively. 

For regression adjustments to be trustworthy standardized differences of means should be less that 
0.25 13 and you want very few of significant t-tests. 14 Checks were conducted for statistical balance 
and modified the models in step 2 by including interactions and higher-order terms of the 
unbalanced characteristics until satisfactory statistical balance was achieved i.e. models were 
improved until it was no longer possible to reduce the number of significant t-tests or standardized 
differences <0.25. For this study total of five models that achieved propensity score balance to 
achieve this result were fit. 

Pre-treatment characteristics and standardized differences between awardees and non-awardees 
within each of the seven strata were examined. The sizes of the treatment-control differences 

11 Dehejia & Wahba, 2002; Agodini, R., & Dynarski, M. 2004. Are experiments the only option? A look at 
dropout prevention programs. Review of Economics and Statistics, 86(1), 180-194. 

12 Morgan, S.L., & Winship, C. 2008. Counterfactuals and causal inference: Methods and principles for social 
Research. New York: Cambridge University Press. 

13 Rubin, D.B. 2001. Using propensity scores to help design observational studies: Application to the tobacco 
litigation. Health Services and Outcomes Research Methodology, 2, 169-188. 

14 Stuart, 2010. 


B 


x,s 



Where: 
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expressed in standard deviation units (standardized differences) ranged from 0.020 to 0.794 (there 
were five standardized differences greater than 0.25), with the largest differences in the block with 
the smallest awardee group size (block 1). Controls were included the impact model for any 
characteristics in the impact models whose standardized differences were greater than 0.25. 


Impact Models 


Following the propensity score matching, the impact of the IRFP program was estimated by 
comparing fellows' outcomes to those of their comparison group to determine what fellows' 
expected outcomes would have been had they not received an IRFP award. 

Estimation of Impacts 

After creating the propensity score strata, a multivariable regression model was used to estimate 
the impact of the program of interest. This regression model employed a number of matching 
characteristics and other variables (see Exhibit C.6) that are hypothesized to affect the outcomes of 
interest such as covariates. The inclusion of the matching characteristics in this model give us the 
chance to get a "doubly-robust" impact estimate since they would have been used twice: both in the 
propensity score model and in the estimation of impacts. 15 The following regression model was used 
to estimate the program impact: 16 

Impact Model 

[i] r, = A +/W +Z/V/ +ZA-TS,- + f l A^ x i + s , 

j = 1 j = 1 n = 1 

Where: 

Y t is the outcome of interest for individual i , 

T ( is the treatment indicator for individual i (l=awardee , 0=non-awardee), 

S/ is the indicator (dummy) variable for the j ,h propensity score stratum. The prototypical 
model includes the total number of strata (b) minus one strata indicators (j=l,2 ,..., b-1). The last 
stratum is the reference stratum and a dummy for this stratum is not included in the model, 

X" is the n (n=l,2,...,N) covariate for individual i (such as gender, URM, etc.) that are grand- 
mean centered, and 

£ j is the usual error term for individual Interpretation of parameters 


Flo, D.E., Imai, K., King, G., & Stuart, E. A. 2007. Matching as nonparametric preprocessing for reducing 
model dependence in parametric causal inference. Political Analysis, 15, 199-236.; Morgan S.L., & 
Harding, D. J. 2006. Matching estimators of causal effects: Prospects and pitfalls in theory and practice. 
Sociological Methods & Research, 35(1), 3-60. 

16 For illustrative purposes, the impact model is presented for continuous outcomes. For binary outcomes, a 
logistic model was fit structured similarly to the model in Equation 1. 
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Interpretation of the coefficients in the model is as follows: 

P 0 is the covariate-adjusted mean value of the outcome for the non-awardees in the reference 
propensity score stratum, 

/v /V 

J3 0 + +i (j=l,2,...,b-l) is the covariate-adjusted mean value of the outcome for the non- 

•th 

awardees in the j stratum, 

/?, is the impact estimate for the reference stratum (i.e. the difference between the mean value 
of the outcome of the awardees and non-awardees in the reference stratum), 

a. , 

P j+b (j=l,2,...,b-l) is the difference between the impact estimate for the /"stratum and the 
impact estimate for the reference stratum, 

Pi + ft j+b (j=l,2,...,b-l) is the impact estimate (i.e., the covariate adjusted difference between 
the outcomes of the awardees and non-awardees) for the j' 1 ' stratum, and 

P n+(b+ 3 ) (n=l,2,...,N) is the estimated overall relationship between the n covariate and the 
outcome controlling for other covariates. 

Treatment Effects 

Overall treatment effect (estimated impact) 

As seen, the model in Equation 1 allows for the estimation of separate treatment effect estimates 
for each propensity score stratum. More specifically, p l + ft j+b (j=l,2,...,b-l) is the impact estimate 

for the j th (j=l, 2,..., b-1) stratum. In order to calculate an overall treatment effect estimate, the 
stratum-specific estimates are aggregated as follows 17 : 

[2] TE = YjPj (A + A+b) + p , A ; where 

7=1 

n ■ 

P is the proportion of treatment group members in the /" stratum (i.e. — where N is the 
1 N ' 

total number of awardees in the sample and rij is the number of awardees in the j th stratum) , 

and 


P r is the proportion of treatment group members in the reference stratum. 


Stratum-specific treatment effect estimates can be aggregated to yield an overall impact estimate in a 
number of ways. The method chosen here— weighing the estimate for each stratum by the proportion of 
treatment group members in that stratum— is widely used (Morgan & Harding, 2006; Caliendo & 
Kopeinig, 2007). 
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Overall covariate-adjusted mean for non-awardees 

b - 1 

[3] 

^ AdjNonAwardees = Z p j (A + A+i ) + p r A ; where 
y=i 

n J 

p 

j is the proportion of treatment group members in the j th stratum (i.e. N / where N is the 
total number of awardees in the sample and nj is the number of awardees in the j th stratum) , 
and 


is the proportion of treatment group members in the reference stratum. 


Overall Covariate-Adjusted Mean for Awardees 


[4] 


" AdjAwardeesr 


- v +TE 

AdjNonAwardees 


Estimated coefficients from the impact model and the overall impact estimate are presented as well 
as their corresponding standard errors and p-values. Hence, for dichotomous outcomes, impact 
estimates are presented in the form of percentage points. For continuous outcomes, overall impact 
estimates in "effect size" units (e.g., Hedges' g) are also presented. The effect size for an impact 
estimate was calculated as: 


[5] 


ES 


TE 

PooledSD 


Where 


TE is calculated as shown in Equation 2, and 


[ 6 ] 


PooledSD = 


l (N,-l)S t 2 + (N c -l)S; 
V (N,-1) + (N C -1) 


Where 


N t =sample size of treatment group, 

=sample size of comparison group, 

Sf =variance of the outcome for treatment group (unadjusted), and 
S l =variance of the outcome for comparison group (unadjusted). 
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Exhibit C.6: Pre-Award Data Used To Construct Comparable Groups of IRFP Fellows and Unfunded Applicants for Impact Models 


Covariate 

Reason for Inclusion in Impact Model 

Data Source(s) 

Type 

Definition 

Mean proposal score 

Proposal score indicates quality of application: 
Characteristic was not balanced in block 7 

NSF Extant Data 

Continuous 

Average score across reviews 
(1-5) 

Gender 

Control for gender differences 

IFRP Applicant Survey: 
G1 

Dichotomous 

l=Female 

0=Male 

Under-represented 
minority status (URM) 

Preference given for under-represented 
minority status (members of these groups were 
encouraged to apply): Characteristic was not 
balanced in block 4 and 6 

IFRP Applicant Survey: 
G2, G3 

Dichotomous 

l=Other race(s)/ethnicity 
0=Asian Only or White Only 

Under-represented 
minority status 
imputation flag 

Flag to represent missing data pattern for URM 
variable: URM was not balanced in block 4 and 
6 

Abt Created 

Dichotomous 

l=imputed 
0=not imputed 

South Central America 

Application was to South Central America: 
Characteristic was not balanced in block 2 

NSF Extant Data 

Dichotomous 

l=South/Central America 
0=Rest of the world 

Number of years since 
completing PhD 

Control for number of years since completing 
PhD 

IRFP Applicant Survey: 
A4a 

Continuous 

Number of years (0-23) 

Applicant had a 
Biological, agricultural, 
or environmental life 
sciences degree 

Applicants field of study was biological, 
agricultural, or environmental life sciences: 
Characteristic was not balanced in block 2 

IFRP Applicant Survey: 
A5 

Dichotomous 

l=Biological, agricultural, or 
environmental life sciences 
0=Rest of disciplines 

Prior international 
exposure 

Control for prior exposure to foreign colleagues 
or former program fellow favorable to 
applicant: Characteristic was not balanced in 
block 2 

IFRP Applicant Survey: 
C4 

Dichotomous 

l=Prior exposure to foreign 
colleagues or former program 
fellow 

0=No prior exposure to foreign 
colleagues or former program 
fellow 

Total pre-award 
publications 

Control for prior number of publications used 
only in the model that examines post award 
publications 

IFRP Applicant Survey: 
C7 

Continuous 

Total pre-award publication 
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Exhibit C.6: Pre-Award Data Used To Construct Comparable Groups of IRFP Fellows and Unfunded Applicants for Impact Models 


Covariate 

Reason for Inclusion in Impact Model 

Data Source(s) 

Type 

Definition 

Field of study 

Control for applicants field od study 

IFRP Applicant Survey: 
A5 

Dichotomous 

l=Biological, agricultural, or 
environmental life sciences 
0=AII other disciplines 


l=Physical and related sciences 
(includes Chemistry except 
biochemistry; earth, 
atmospheric, ocean sciences; 
physics) 

0=AII other disciplines 

l=Computer and information 
sciences & Mathematics and 
statistics 

0=All other disciplines 

l=Psychology & social sciences 
&related science 
0=AII other disciplines 

l=Engineering 
0=AII other disciplines 

l=Health (non-STEM) 

0=AII other disciplines 


Abt Associates Inc. 


Appendix C C-30 




Evaluation of International Research Fellowship Program: Final Report 


Sensitivity Analysis 

A series of sensitivity analyses were conducted to ensure that the impact results reported are 
reliable and robust. In particular, the following sensitivity analyses were conducted: 

Using a different number of strata in the propensity score stratification 

As a sensitivity analysis, propensity blocks were combined and the impact estimate was recalculated 
using impact model [ 1] above. Blocks 1 and 2 were combined (blocks with a small number of 
treatment group members relative to the number of comparison group members) and blocks 6 and 
7 (blocks with a small number of comparison group members relative to the number of treatment 
group members). 

Check sensitivity to matching method used 

Stratification (also called interval matching) was the primary method for matching fellows and 
unfunded applicants. To check the sensitivity of the estimate to the matching method used, impacts 
were also estimated using the following propensity score matching methods: one-to-one (1:1), K-to- 
one (3:1), and radius (caliper) matching. 18 In its simplest form, 1:1 matching selects for each treated 
individual(s) the control individual with the closest propensity score (matching each fellow(s) with 
the most similar non-fellow). K:1 and radius matching improves on 1:1 matching by restricting the 
distance of the matches. All of the matches are then pooled into matched treated and control 
groups and analyses were run using groups as a whole rather than as individual matched pairs. 

These methods discard treatment/comparison cases without matches which could potentially lead 
to a reduction in power 19 but could lead to higher precision. For the K:1 and radius matching weights 
were used in the analysis to represent the pairing. The following model was then used to estimate 
the impact for this matching method: 

[10 ]Y = p o +0 l (T i ) + f i 0 n+l X?+e i 

n= 1 

Where: 


K is the outcome of interest for individual i, 

T t is the treatment indicator for individual i (l=awardee , 0=non-awardee), 

X" is the n"' (n=l, 2,..., N) covariate for individual i (such as gender, age, etc.) that are grand- 
mean centered, and 


For the radius match a caliper of 0.005 was used. 

19 The reduction in power is minimal, for two reasons. First, in a two sample comparison of means, the 
precision is largely driven by the smaller group size (Cohen, 1988). So if the treatment groups stays the 
same size and only the control group decreases size, the overall power may not actually be reduced very 
much (Flo et al., 2007). Second, the power increases when the groups are more similar because of the 
reduced extrapolation and higher precision that is obtained when comparing groups that are similar 
versus groups that are different. Snedecor, G. W. and Cochran, W. G. 1980. Statistical Methods, 7th ed. 
Iowa State Univ. Press, Ames, IA. MR0614143. 
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s j is the usual error term for individual i. 

P Q is the covariate-adjusted mean value of the outcome for the non-awardees, 

/?, is the overall impact estimate (i.e. the difference between the mean value of the outcome of 
the awardees and non-awardees ), 

P 0 + p x is the covariate-adjusted mean value of the outcome for the awardees, and 

A. 

Pn + ( b - i) (n=l,2,...,N) is the estimated overall relationship between the n covariate and the 
outcome controlling for other covariates. 

Sensitivity Checks Conclusion 

Sensitivity checks were conducted to determine how confident one could be with the results from 
the primary method of matching, i.e., to determine how sensitive the findings are to the different 
matching methods used. 

Exhibit C.7 shows the overall standardized impact estimate and a 95 percent confidence interval for 
the estimate for each of the outcomes from the primary method of matching. It also shows the 
estimate and 95 percent confidence interval for the impact estimate for each of the outcomes for 
the other matching techniques. This exhibit shows that while the impact estimates are robust (i.e. 
they are the same across matching methods) the estimates from the primary matching method are 
sometimes less precise (larger confidence intervals) then the estimates from other matching 
techniques. This is due to larger standard errors that occur because of the imbalance in the number 
of treatment and control members in some of the matched blocks in the primary method. Results 
from the 5-block method are reported since this method is similar to the primary method and 
increases precision. Precision is improved in the 5-block method because the balance in the number 
of treatment and control members in the matched blocks is improved. 
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Exhibit C.7: Overall Impact Estimates and Estimates with Each Sensitivity Analysis 
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Currently has tenure 


Total number of post-award publications 
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Total # of Inti. Postdocs 


Work with Individuals in Other Countries 


03 

CO 



03 

CO 



# of Pubs w/ Foreign Collab. 


Prop, of Pubs w/ Foreign Collab. 


03 
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£ 
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Leadership in Fostering Inti Collab. 


Duration of employment outside the U.S. 



Primary 5 block 


1 to 1 
Method 


3to1 


radius 


Type of current work with foreign individuals 



EXHIBIT SHOWS: On the far left of each graph, the impact estimate and 95% confidence interval is shown for the primary 
matching method (stratification). Moving from left to right, the graphs show the impact estimate and 95% confidence 
interval for the different matching methods. 

NOTE: If the circle is above the dashed line representing 0.0 then the treatment effect is positive, if below 0 then the 
treatment effect is negative. If the confidence interval does not include 0 then the finding is significant. 
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Appendix D: Detailed Description of Benchmarking Analysis 


This appendix describes the methodology used to compare the IRFP applicants and awardees and 
applicants to those of a nationally-representative sample. 

The 2006 and 2008 Survey of Doctoral Recipients (SDR) was utilized in this study to compare the 
IRFP fellows and applicants to those of a nationally-representative sample of science, engineering 
and health (SEH) doctoral degree recipients on key employment, postdoctoral appointment, and 
international collaboration variables. 

For this study, the primary comparison group for IRFP fellows is a propensity-score matched sample 
of unfunded IRFP applicants. The SDR respondents were used as a secondary comparison group to 
assess how fellows' and all applicants' outcome indicators compare to national averages. SDR 
indicators most relevant to the study include employment sector and current position, and the 
nature and extent of collaboration with foreign researchers. For example, comparisons included the 
percentage of respondents employed by academic institutions in the sample of IRFP fellows and 
applicants versus the national samples (Question D2a in 2008 SDR). 

Exhibit D.l shows the applicant survey items that were designed to allow comparison of IRFP 
awardees to SDR respondents. 
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Exhibit D.l: IRFP Applicant Survey Items Comparable to Items on the 2006 Survey of Doctoral 


Recipients 

IRFP 

Applicant 

Survey 

Item 

SDR Item 

Item text 

SDR 

Dla 

A38 

SDR 2006 

Since completing your first 
doctoral degree, how many post- 
docs, if any, have you held? 1 

Number of postdocs 
**IRFP** 

Var=number of postdocs (0,1,2,...) 


** SDR** 

If 'none' box checked then var=0; 

Else Var=number of postdocs (0,1,2,...) 

Report average number of postdocs. 


Control Variables: 

The number of years between 
receipt of first PhD degree and the 
reporting year of outcomes 
Field of study for the first PhD 
URM 


Die A1 During the week of [reference 

SDR 2008 week] 2 were you working for pay 
or profit? (includes postdoctoral 
appointments) 


Gender 

Working during reference week. 
**IRFP** 

If Die in (1,2) then var=l; 

If Die in (3) then var=0; 


**SDR** 

If Al=l then var=l; 
If Al=2 then var=0; 


D2 A12 Was your primary employer 

SDR 2008 during the week of [reference 

week] an educational institution? 


Report percent answering 'Yes' (1). 

Control Variables: 

The number of years between 
receipt of first PhD degree and the 
reporting year of outcomes 
Field of study for the first PhD 
URM 
Gender 

**IRFP** 

If D2=l then var=l; 

If D2=2 then var=0; 


**SDR** 

If A12=l then var=l; 
If A12=2 then var=0; 


Report percent answering 'Yes' (1). 

Control Variables: 

The number of years between 
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Exhibit D.l: IRFP Applicant Survey Items Comparable to Items on the 2006 Survey of Doctoral 
Recipients 


IRFP 




Applicant 




Survey 




Item 

SDR Item 

Item text 

SDR 


receipt of first PhD degree and the 


reporting year of outcomes 
Field of study for the first PhD 
URM 


Gender 


D3 


All Which one of the following best 

SDR 2008 describes your primary employer 
during [reference week]? 
Self-Employed or a Business 
Owner 

Private Sector 

Local Government 

State Government 

U.S Federal Government 

U.S Military 

Other 


Employment Type: 

**IRFP** 

Variables coded as: 
l=Yes, 0=No 

Report percent answering 'Yes' (1) to 
the following positions: 

Self-Employed or a Business 
Owner 

If D2=0 and D3=l then var=l; 
0=else (note D2=l is included 
in this else) 

Private Sector 

If D2=0 and D3=2 then var=l; 
0=else (note D2=l is included 
in this else) 

Local Government or State 
Government 

If D2=0 and D3 in (3,4) then 
var=l; 0=else (note D2=l is 
included in this else) 

U.S Federal Government or U.S 
Military 

If D2=0 and D3 in (5,6) then 
var=l; 0=else (note D2=l is 
included in this else) 

Other 

If D2=0 and D3 in (7) then 
var=l; 0=else (note D2=l is 
included in this else) 


**SDR** 

Variables coded as: 
l=Yes, 0=No 

Report percent answering 'Yes' (1) to 
the following positions: 

Self-Employed or a Business 
Owner 

All in (1, 2) & A12=2 then 
var=l; 

All in (3, 4, 5, 6, 7, 8, 9) & 
A12=2 then var=0; 
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Exhibit D.l: IRFP Applicant Survey Items Comparable to Items on the 2006 Survey of Doctoral 
Recipients 


IRFP 




Applicant 




Survey 




Item 

SDR Item 

Item text 

SDR 


D2a A13 Currently employed as research 

SDR 2008 faculty at 4-year 

college/university, medical 
school, or university-affiliated 
research 


Private Sector 
All in (3,4) &A12=2 then 
var=l; All in (1, 2, 5, 6, 7, 8, 9) 
& A12=2 then var=0; 

Local Government 

All in (5, 6) & A12=2 then 

var=l; 

All in (3, 4, 1, 2, 7, 8, 9) & 
A12=2 then var=0; 

U.S Federal Government or U.S 
Military 

All in (7, 8) & A12=2 then 
var=l; 

All in (3, 4, 1, 2, 5, 6, 9) & 
A12=2 then var=0; 

Other 

All in (9) & A12=2 then var=l; 
All in (3, 4, 1, 2, 7, 8, 9, 5, 6) 

& A12=2 then var=0; 

Currently employed at: 

**IRFP** 

If D2a in (3,4,5) then var=l 
If D2a in (1,2,6) then var=0 

**SDR** 

If A13 in (3,4,5) then var=l 
If A13 in (1,2,6) then var=0 


Report percent answering 'Yes' (1). 

Control Variables: 

The number of years between 
receipt of first PhD degree and the 
reporting year of outcomes 
Field of study for the first PhD 
URM 
Gender 


D2b 5 

A14 5 

During the week of [reference 

Academic Position 


SDR 2008 

week] what type of academic 
position did you hold at this 
institution? 6 

President, Provost, or Chancellor 
(any level) 

Dean (any level), department 
head or chair 

Research faculty, scientist, 

Variables coded as: 
l=Yes, 0=No 

Report percent answering 'Yes' (1) to 
the following positions: 

President, Provost, or Chancellor (any 
level) 

Dean (any level), department head or 
chair 
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Exhibit D.l: IRFP Applicant Survey Items Comparable to Items on the 2006 Survey of Doctoral 
Recipients 


IRFP 

Applicant 

Survey 

Item 

SDR Item 

Item text 

SDR 



associate or fellow 
Teaching faculty 
Adjunct faculty 
Other (please specify): 

Research faculty, scientist, associate 

or fellow 

Teaching faculty 

Adjunct faculty 

Other (please specify) 




Control Variables: 

The number of years between 
receipt of first PhD degree and the 
reporting year of outcomes 
Field of study for the first PhD 
URM 
Gender 

D2c 5 

A15 5 

SDR 2008 

Currently has faculty rank of 
Assistant, Associate or Full 

Faculty Rank: 


Professor **| RFP ** 

If D2c in (3,4,5) then var=l 
If D2c in (6,7,8) then var=0 

**SDR** 

If A15 in (3,4,5) then var=l 
If A15 in (6,7,8) then var=0 

Report percent answering 'Yes' (1). 

Control Variables: 

The number of years between 
receipt of first PhD degree and the 
reporting year of outcomes 
Field of study for the first PhD 
URM 
Gender. 


D2d 5 

A16 5 

What was your tenure status? 

Tenure Status: 


SDR 2008 


**| RFP ** 

If D2d=3 then var=l 
If D2d in (4,5) then var=0 




**SDR** 

If A16=3 then var=l 
If A16 in (4,5) then var=0 




Report percent answering 'Yes' (1). 




Control Variables: 
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Exhibit D.l: IRFP Applicant Survey Items Comparable to Items on the 2006 Survey of Doctoral 
Recipients 


IRFP 

Applicant 

Survey 

Item SDR Item Item text SDR 


The number of years between 
receipt of first PhD degree and the 
reporting year of outcomes 
Field of study for the first PhD 
URM 
Gender 

D4 A27, Item 4 In performing the primary job you Work with individuals outside of US 

SDR 2006 held during the week of [reference Variable coded as 1=765, 0=No 
week], did you work with Report percent answering "yes" 

individuals located in countries 

other than the US? Report percent answering 'Yes' (1). 


Control Variables: 

The number of years between 
receipt of first PhD degree and the 
reporting year of outcomes 
Field of study for the first PhD 
URM 


Gender 


D6, item 

Cl, Item 2 

IRFP: Peer-reviewed Journal 

Number of journal articles published 

1 

SDR 2008 

articles 

* * | ppp* * 

Var=number of articles(0,l,2, ) 



SDR: Articles, (co)authored by you, 
have 

**SDR** 



been accepted for publication in a 
refereed professional journal? 

Var=number of articles(0, 1,2, ) 




Control Variables: 

Field of study for the first PhD 

URM 

Gender 

IRFP only: The number of years 




between application to IRFP and 
survey reference date. Everyone in 
the SDR sample reported 
publication in the last 5 years this 
was not a control. 


D6, item Cl, item 1 IRFP: Peer-reviewed conference 

2 SDR 2008 publications (e.g. abstracts, 

conference papers, posters) 


Number of conference publications 

* * IRFP** 

Var=number of conference 
publications^, 1,2, ) 


SDR: Papers have you 
(co)authored for 

presentation at regional, national 
or 

international conferences? (Do 
not count presentations of the 


**SDR** 

Var=number of conference 
publications^, 1,2, ) 

Report percent answering 'Yes' (1) 


Abt Associates Inc. 


Appendix D D-6 




Evaluation of International Research Fellowship Program: Final Report 


Exhibit D.l: IRFP Applicant Survey Items Comparable to Items on the 2006 Survey of Doctoral 
Recipients 


IRFP 

Applicant 

Survey 

Item 

SDR Item 

Item text 

SDR 

same work more than once.) 


Control Variables: 

Field of study for the first PhD 

URM 

Gender 

IRFP only: The number of years 
between application to IRFP and 
survey reference date. Everyone in 
the SDR sample reported 
publication in the last 5 years so 
this was not a control. 


D6, item 

C2, C3, item 1 

IRFP: Patents, registered or 

Number of patents 

3 

and 2 

pending 

* * IRFP** 


SDR 2008 

SDR: Since October 2003, have you 

Var=number of patents(0,l,2, ) 



been named as an 

inventor on any application for a 

U.S. patent? 

or 

Flow many applications for U.S. 

**SDR** 

If C2=2 then var=0 
Else Var=sum(C3.1:C3.2) 



Control Variables: 



patents have named you as an 

Field of study for the first PhD 



inventor? 

URM 



or 

Gender 



Flow many U.S. patents have 

IRFP only: The number of years 



been granted to you as an 

between application to IRFP and 



inventor? 

survey reference date. Everyone in 
the SDR sample reported 
publication in the last 5 years so 
this was not a control. 

D6, item 

Cl, item 3 

IRFP: Book Chapter(s) 

Number of book chapter(s) 

4 

SDR 2008 

SDR: Books or monographs, 

* * IRFP** 

Var=number of chapters(0,l,2, ) 



(co)authored 

by you, have been published or 

**SDR** 



accepted for publication? 

Var=number of chapters(0,l,2, ) 


Control Variables: 

Field of study for the first PhD 

URM 

Gender 

IRFP only: The number of years 
between application to IRFP and 
survey reference date. Everyone in 
the SDR sample reported 
publication in the last 5 years so 
this was not a control. 
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Exhibit D.l: IRFP Applicant Survey Items Comparable to Items on the 2006 Survey of Doctoral 
Recipients 


IRFP 




Applicant 




Survey 




Item 

SDR Item 

Item text 

SDR 


NOTES: 


1 The wording for this question is slightly different between IRFP and SDR. IRFP is: Since receiving your first doctoral 
degree how many postdoctoral appointments have you held? Please include any postdocs you held through October 
1, 2010. SDR is: Since completing your first doctoral degree, how many "postdocs," if any, have you held? Please 
include any postdocs you held through April 1, 2006. 

2 The reference week for the 2008 SDR was October 1, 2008; for the IRFP applicant survey, October 1, 2010. 

3 The IRFP question asks about primary employer and SDR asks about principle employer. 

4 For IRFP, D2a is presented only if D2=Yes. For the 2006 SDR, Item A13 applies only if A12=Yes. 

5 For IRFP, this item is only presented if D2=Yes and D2a not equal to preschool, elementary, middle, secondary 
school; for the 2006 SDR, this item applies only if A12=yes and A13 not equal to preschool, elementary, middle, 
secondary school 

6 Items have different response options. In IRFP Research Assistant, Teaching Assistant, and Postdoc were collapsed 
into "other." Postdocs would skip out of D2b, so IFRP did not have options for Research Assistant, or Teaching 
Assistant. For the SDR data Research Assistant, Teaching Assistant were grouped into the Other category. 


Limitations of These Comparisons 


There were four noteworthy limitations of these data. First, the SDR survey was not designed to 
measure many of the outcomes that are pertinent to this study. Research productivity, for example, 
is a particularly notable omission. The use of the SDR as a national comparison is limited to the 
subset of items such as current employment, number of international collaborations, and a few 
others. 

The second limitation of the SDR data is the difference in timing, as this study collected data in 2011, 
whereas the SDR data come from surveys administered in 2006 and 2008. As a result, when the year 
of first doctoral degree attainment was used as a variable on which to compare the study population 
to the SDR population, the groups are out of phase by two to five years, depending on the SDR cycle. 
For example, 2006 SDR respondents who earned their PhDs in 2000 would have had six years to 
achieve their outcomes by the time of the data collection. In contrast, respondents to the IRFP 
survey who earned their PhD in the same year (2000) would have had 11 years to achieve outcomes, 
possibly biasing any comparison of the two groups. To mitigate this problem, comparisons 
controlled for the number of years between the receipt of first PhD degree and the reporting year of 
outcomes in the analyses. 

The third limitation is that the sampling frame for the SDR excludes individuals living outside of the 
United States during the survey reference period. This methodology creates a sample bias relevant 
to the IRFP program. The sampling bias is compounded in that individuals excluded from the initial 
SDR would be excluded from any subsequent follow-up SDR surveys. As a result, the SDR sample 
might include fewer individuals likely to be engaged in international research collaboration. 

Finally, the SDR sample has the potential to include some IRFP recipients. It was not possible to 
obtain personal identifying information for SDR respondents, so IRFP participants could not be 
removed from the SDR sample. The 2006 SDR sample consisted of 30,817 individuals. The 2008 SDR 
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sample consisted of 29,974 individuals. Since there are only 457 IRP awardees in the sample, the 
potential overlap is very small (<1 percent), hence these were treated as independent samples. 


Performing the Calculations 


The goal of the benchmarking calculation was perform a test of whether the difference between the 
two adjusted means from the two samples is equal to zero. Exhibit D.2 shows a specification of the 
hypothesis test and the method for calculating the p-value from the test is shown. Exhibit D.3 shows 
a method for calculating the variance (and standard error) of the difference of the means and 
Exhibit D.4 shows a method for calculating the variance (and standard error) of the difference of the 
proportions. 20 

Exhibit D.2: Hypothesis Testing 

Let Xj denote the estimated mean from the IFRP sample of size 
X-, denote the mean from the SDR sample of size n 2 . 

SE(x 2 — Tj) denote the standard error of the difference between the two sample means. 

H o :x l -x 2 =0 vs H a : x { - x 2 ^ 0 

Test Statistic is: 

(*i -x 2 )-0 

SE(x j -x 2 ) 

If the observed value of t as calculated above is greater than the critical value from the t-distribution with 
n -2 degrees of freedom and a=0.05, the null hypothesis will be rejected at the p<0.05 level. 

OR 

P Yl 

Let denote the estimated proportion from the IFRP sample of size '. 

n yi 

^ 2 denote the estimated proportion from the SDR sample of size 2 . 

SE(p { — p~,) denote the standard error of the difference between the two sample proportion. 

H o ■ Pi - Pi = 0 ™ H a : Pl -p 2 *0 

Let the z be the standardized deviate, calculated as: 

(Pi-P 2 )-° 

SEip^-pJ 

Compare z to the quantiles of a standard normal distribution, N(0,1), to find the two-sided probability of 
obtaining a deviate with absolute value that is as large or larger than z. If the absolute value of z is greater 
than 1.96, the null hypothesis will be rejected at the p<0.05 level. 


20 Kish, L. 1995. Survey sampling. New York: Wiley. 
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Exhibit D.3. Estimating the Variance of the Difference in Two Sample Means 


Let Xj denote the estimated mean from the IFRP sample of size 1 \ . Let X 2 denote the mean from the SDR 

sample of size tl 2 . The difference between the two sample means was tested. The estimated variance of 
the difference between the two sample mean is written as 


v(Xj —x 2 ) = v(x { ) + v(x 2 ) - 2cov(x 1 , x 2 ) 


Under simple random sampling, the variance of the difference can be written as 


v(x, - x 2 ) = v(x l ) + v(x 2 ) - 


2 /V 2 W v (* 1 )v(x 2 ) 

V"i n 2 


v(x ) vi v(x 1 

v 1 ' is the estimated variance of the mean based on the IRFP sample of 1 units, v 2 ' is the estimated 

yi 

variance of the SDR mean based on 2 units and m is the amount of overlap between the two samples. 

The correlation ( ^ x ' Xl ) is estimated based on the overlap. Since m is almost 0 the variance can be 
estimated as: 


v(x t — x 2 ) = v(Xj ) + v(x 2 ) 

The variance under the sample 21 design is obtained from proc survey reg for the first mean and the second 
mean. The square root of the variance gives the standard error of the difference in the two means, which 
can be used in a statistical test. 


21 


The IRFP was treated as a simple random sample or "pseudo-sample" since awardees from a different 
grant period might result in a different estimate. 
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Exhibit D.4: Estimating the Variance of the Difference in Two Sample Proportions Based on 
Independent 22 Samples. 


Let denote the estimated proportion from the IFRP sample of size 11 1 . Let P 2 denote the 

proportion from the SDR sample of size 2 . The difference between the two sample means 
was tested. The estimated variance of the difference between the two sample mean is written 
as 

v(Pi -P 2 ) = v (Pi ) + V (P 2 ) - 2mcov(/?, ,p 2 ), 

Since m is almost 0 one can write the estimated variance of the difference between the two 
proportions as: 

v(p l -p 2 )= v (Pi) + v(p 2 ) 

One can get the variance under the sample 23 design from proc survey reg for the first 
proportion and the second proportion. The square root of the variance gives the standard error 
of the difference in the two means, which can be used in a statistical test. 


Estimation of Mean and Variance 


As mentioned above SAS proc survey reg was used to obtain the adjusted mean and standard error. 
The following regression model was used to estimate the adjusted mean and standard error for 
each sample: 

t =A,+Z/W+*, 

n=l 


Where: 


Yj is the outcome of interest for individual i , 

X" is the n"‘ (n=l,2,...,N) covariate for individual i (such as gender, URM, number of years 
between PHD and survey date, etc.) that are grand-mean centered across the samples, and 

£■ is the usual error term for individual i. 


Interpretation of Parameters 


Interpretation of the coefficients in the model is as follows: 


The IRFP sample is a potentially a very small proportion (close to 0) of the SDR sample so these were 
treated as independent samples. 

23 The IRFP is treated as a simple random sample as "pseudo-sample" since awardees from a different grant 
period might produce a different estimate. 


Abt Associates Inc. 


Appendix D D-11 






Evaluation of International Research Fellowship Program: Final Report 


J3 0 is the covariate-adjusted mean value of the outcome for the sample, 

. 

J3 n (n=l,2,...,N) is the estimated overall relationship between the n covariate and the outcome 
controlling for other covariates. 
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Appendix E: Surveys 
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International Research Fellowship Program (IRFP) 

Applicant Survey Items 


Applicant Survey 


1 


Programming notes appear in red or brown text throughout. 

Hyperlinked text will aooear tffllfeQSSidl and UiftdteiHlijifttdl in this document but should not be formatted 
this way onscreen unless noted (underlining is fine except for email addresses, but embossing is not). 

General guidelines: 

Each screen displayed one question at a time unless otherwise indicated. 

Each screen displayed a standard set of Navigaton buttons shown below. 



Some items contained placeholders for fields that were programmed to be auto-filled from the sample 
file for each respondent. These fields are enclosed in [brackets] and printed in brown. Some of these 
placeholders used internal (control) variables. For example, [year of application + 2] were calculated 
based on a sample field called [year of application]. If this field = 1999 for a respondent then the 
resulting auto-filled [year of application + 2] would equal 2001. 

Textboxes for open-ended responses are indicated with [textbox, 150] where the number indicates the 
length of the field (how many characters the respondent may type). 

REMINDERS for BLANK RESPONSES and DEFAULTS for SKIP PATTERNS if RESPONSE was MISSING 

• No items required a response. 

• No items required a reminder to enter an answer/select a response unless indicated below. If a 
reminder was needed for the first time user leaves an item blank, the item was re-displayed 
adding the text below shown on-screen in bold, red text. 

These reminders and defaults are given at the end of this document. 
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Sample variables. 


AwdStatus: Fellow or Unfunded Applicant: Two groups of respondents will be tracked using the 
sample file: Fellows, and Unfunded Applicants. In the sample file, Awardee = FELLOW and Declinee = 
UNFUNDED APPLICANT. The outcome of the most recent year of application for individuals with 
multiple IRFP applicants was used to determine Award Status (AwdStatus) 


PI_Name will consist of First Name, Middle Name, Last Name. Note that Middle Name may be an 
initial or may be blank. 

Phd_year: year in which respondent received his/her doctoral degree. If this field is blank or missing it 
should be auto-filled with the following text string: Unknown 

Year of application: Indicates the cohort (year) in which the respondent applied for participation. 
Range: 1992 through 2009. The most recent year of application for individuals with multiple IRFP 
applicants was used. 

IRFP country: The name of the nation where the respondent wanted to pursue the IRFP fellowship. 
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Introductory screen: 


Survey of former applicants to the 
National Science Foundation’s 
International Research Fellowship Program (IRFP) 


Welcome and thank for your interest in this study. This survey is being conducted by Abt Associates 
Inc., and our subsidiary, AbtSRBI, for the National Science Foundation (NSF), to learn about individuals 
who applied to NSF's International Research Fellowship Program (IRFP) for a postdoctoral fellowship . 
This survey will give NSF information about the professional characteristics and international 
collaborations of U.S. scientists and engineers and help NSF improve programs intended to foster a 
globally engaged scientific and engineering workforce. You are receiving this survey because you are 
listed in the IRFP database as a former applicant to IRFP. We estimate that it will take approximately 
30 minutes to complete the survey. 


Confidentiality and Participation 

Participation in the survey is voluntary and nonparticipation will have no impact on you or your 
institution. You may skip questions on the survey or discontinue participation at any time. There are 
minimal risks associated with your participation. We take your privacy very seriously. Your responses 
to this survey will be protected under the Privacy Act. There is minimal risk of breach of 
confidentiality, and we have put in place procedures to minimize this risk. Reports will never identify 
you by name, and information from the study will only be reported in the aggregate at the program 
level, combined with about 250 other responses. When we receive your survey we will detach and 
store separately your name and other identifying information that could be used to link you to your 
survey responses. Survey responses will be stored on a secure drive that is only accessible to 
members on the study team. Only study team web technicians and data analysts from Abt Associates 
and AbtSRBI will see individual responses that can be linked to you. Survey data files will be shared 
with NSF at the end of the study, only after study team members have examined the data to be free of 
any information that could help identify you; this cleaning includes procedures to limit someone from 
inferring your identity by analyzing non-identifying data. Flence, we encourage you to respond 
candidly about your experiences. Separate from your individual responses to the survey we will 
provide NSF any updated contact information we have found or requested from you. None of this 
contact information will be linked in any way to your survey responses. At the conclusion of the study, 
Abt Associates and AbtSRBI will destroy all records, electronic or otherwise, that link you to your 
survey responses. 




This study's IRB approval number is #0494, valid from 8/6/2010 to 8/5/2011. For questions, please contact 
Teresa Doksum, IRB Administrator, Abt Associates, at IRB@abtassoc.com. 

The valid OMB control no. for this information collection is 3145-0214. (Expires on 12/31/13) 
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Next screen: 


IRFP applicant survey 


Questions 

If you have questions about the study, please contact the study director, Alina Martinez of Abt 
Associates Inc. at (866) 421-6223 (toll free within the U.S.) or email her at!KFP = SUfV0y#alfc1tass©&e@m 
You may also contact the evaluation's program officer at NSF, John Tsapogas ftsap©t0#ift&f-.g©V). If 
you have questions about your rights as a research participant, you may contact Teresa Doksum, the 
Abt Institutional Review Board Administrator at (877) 520-6835 (toll-free within the U.S.) or by email: 
irb@abtassoc.com. To learn more about this study, please refer to the FlWWWitly Asted QwfeSttoftS 
page. 

Navigating through the survey: 

As you work through the survey, your responses are automatically saved. You may change a response 
by clicking on the PREVIOUS ITEM button. Use the NEXT ITEM button to advance to the next 
question. At any time, you may close your browser if you wish to return and finish at a later time. 
When you log back in, the survey will take you to where you left off. On each page of the survey, a 
FAQ button is provided if you have a question during the survey or need information about how to 
contact the survey administrator 

When you have completed the survey, please click on theSUBNHT button at the end of the survey. 
You may submit the survey even if there are some questions that you choose not to answer. 


Consent 

Please click on “Begin" if you agree to participate in this study. BEGIN 
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FAQs are Optional screens, displayed only if R clicks on fWWWtl» Aslted QuosWoftS 


FREQUENTLY ASKED QUESTIONS 

• What is the IntewatioMil Research Mtowshte Pwgram tlWP)? 

% Why are you doing this stw*y? 

% Why tavei been selected tererttetoate in this swvey? 

% Hew tiki vom get my contact infownatieft? 

% How tena witi this survey take to cematete? 

• How wiM yoo nse myewftmwte? 

• Does this stedyhayehwnansnhietts review clearance? 

% Who is fandine the study? 

• Who are Aht Associates \m and AbtSRBI? 

What is the International Research Fellowship Program (IRFP)? 

The International Research Fellowship Program (IRFP) is a National Science Foundation (NSF) program 
that provides support for new doctoral-level scientists and engineers to conduct research in a foreign 
country for a period of 3 to 24 months. 

Why are you doing this study? 

NSF is interested in learning about the characteristics of scientists and engineers who applied for an 
IRFP Fellowship between 1992 and 2009. Information about the domestic and international 
experiences of IRFP participants and non-participants will help NSF understand the usefulness and 
relevance of international research training for scientists and engineers. Information from this study 
will be used to describe the experiences and career paths of new Ph.D. scientists and engineers who 
expressed interest in pursuing an international collaboration. 

Why have I been selected to participate in this survey? 

You have been selected to participate because we have identified you as having applied to the 
International Research Fellowship Program (IRFP). 

How did you get my contact information? 

We identified you from records maintained by the National Science Foundation on prior IRFP applicants. 
We then obtained your contact information through an internet search. 

How long will this survey take to complete? 

We estimate that the survey will take about 30 minutes. 

How will you use my comments? 

Responses from all survey respondents will be used to answer questions about IRFP applicants' early 
career trajectories, the nature of their research collaborations both within and outside of the U.S., and 
their subsequent career paths. 
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Has this study been reviewed and approved by an institutional review board? 

Yes, the study was also approved by Abt Associates' Institutional Review Board. If you have any concerns 
about your participation in this survey, please contact Teresa Doksum, Institutional Review Board 
Administrator at Abt Associates, at (877) 520-6835 or via email a t 

Who is funding the study? 

The study has been funded by the National Science Foundation under contract 
GS-10F-0086K. Abt Associates and AbtSRBI will complete the study. 

Who are Abt Associates Inc and AbtSRBI? 

Abt Associates is an independent research firm headquartered in Cambridge, Massachusetts. AbtSRBI is 
a wholly-owned subsidiary of Abt Associates specializing in large-scale data collection and public opinion 
research. NSF has contracted with Abt Associates and AbtSRBI to design and implement a survey of 
individuals who applied to NSF's International Research Fellowship Program (IRFP). 
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IRFP applicant survey 


MODULE A: VERIFYING INFORMATION ABOUT YOU 

To begin, we'd like to confirm that we've reached the appropriate person. 

Al. Based on the information that you provided to NSF in your application, your name is [First 
Name, Middle Name/initial, Last Name], Is this correct? 

□ No, my name has changed or my name is misspelled above. My name is: [textbox, 75] 

□ No, I'm not the person named above. -> EXIT SCREEN 

□ Yes, this is correct. 

[Former IRFP fellows only] 

A2.1. Based on the information contained in the NSF database, you were a postdoctoral fellow 
as part of the National Science Foundation's International Research Fellowship Program 
starting in [Year of application]. Is this correct? If you participated in IRFP more than 
once, please consider your most recent experience 

□ Yes, this is correct. -> GO TO A3 

□ No, I never participated in IRFP GO TO CONFIRM FELLOWSHIP SCREEN 

□ No, I applied for the IRFP, but withdrew my application before the award decision was 
made -> GO TO EXIT SCREEN 

□ No, I was awarded the IRFP fellowship, but declined it -> GO TO EXIT SCREEN 

□ No, I participated in IRFP, but my fellowship began in: 

Enter year: [textbox, 4] GO TO A3 [Prompt R to correct entry: "Please type a four-digit 
year between 1990 and 2010"] Set Year of application = entered year. 

[Unfunded IRFP applicants only] 

A2.2. Based on the information contained in the NSF database, you applied to the National 
Science Foundation's International Research Fellows Program (IRFP) for a [year of 
application] postdoctoral fellowship, but did not participate in an IRFP fellowship. Is that 
correct? The year shown is the year in which the fellowship was to begin, not the 
application deadline. 

□ Yes, this is correct. -> GO TO A3 

□ No, I applied for an IRFP fellowship to begin in (Enter year): [textbox 4] [Prompt R to 
correct entry: "Please type a four-digit year between 1990 and 2010"] Set Year of 
application = entered year. 

□ No, I did participate in an IRFP fellowship. GO TO CONFIRM FELLOWSHIP SCREEN 

□ No, I never applied for this program. GO TO CONFIRM FELLOWSHIP SCREEN 
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A3. Based on the information in the NSF database, you applied to spend your IRFP fellowship in 
[IRFP country]. Is this correct? 

□ Yes 

□ No, I applied for an IRFP fellowship in (Enter country): [textbox, 50], Set IRFP_country 
= entered country 

A4a. Our records indicate that you received your doctorate in [phdyear]. Is this correct? 

□ Yes, this is correct. ->GOTOA5 

□ No, I earned my degree in (Enter year): [textbox, 4] ->GOTOA5 Prompt R to correct 
entry: "Please type a four-digit year between 1990 and 2010"] Set phdyear = entered 
year 

□ I have more than one doctorate. ->GOTOA4b 

A4b. Please enter the years that you received your first and your most recent doctoral degrees: 
First doctoral degree received in (enter year): [textbox, 4] 

Most recent doctoral degree (enter year): [textbox 4] 


A5. What discipline best characterizes your field of research when you applied for an IRFP 
fellowship (that is, your doctoral field of research)? First, indicate which one of four 
broader areas in which your discipline falls: 

Check one only: 

□ i Sciences (Biological/Life, Chemical, Computer/Information, 

Environmental, Earth/Atmospheric/Oceanographic, 
Geosciences, Mathematical/Statistical, Physics, Psychology) 

□ 2 Social Sciences (Economics, Sociology, Anthropology/Archaelogy, Political 

Science, Geography, Linguistics, other) 

□ 3 Health/Medical 
□ 4 Engineering 

display one of three screens based on choice above: 

1 Science screen 

2 Social Science, Health screen 

3 Social Science, Health screen 

4 Engineering screen 

After R hits SUBMIT, go to Item B1 
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Science Fields 


For Science/Math Education select the primary scientific or mathematical field, or select 
Soeiial SdieifiiefeS oir HtalUth/Mtdlteall and see "OTHER Social Sciences: Education, General" 

To view other fields (Social Science, Health/Medical, Engineering), please select Co back 

te tire frbwwri categories to change your selection. 


Agricultural/food sciences 

O 

Animal sciences 

O 

Plant sciences 


O 

Food sciences/technology 

o 

OTHER agricultural sciences 

Biological sciences 

o 

Biochemistry/biophysics 

o 

Microbiological sciences/immunology 

Bioengineering, see 

o 

Biology, general 

o 

Nutritional sciences 

ENGINEERING, next page 

o 

Botany 

o 

Pharmacology, human/animal 


o 

Cell/molecular biology 

o 

Physiology and pathology, 


o 

Ecology 


human/animal 


o 

Genetics, animal/plant 

o 

Zoology, general 




o 

OTHER Biological sciences 

Environmental life sciences 

o 

Environmental science/studies 

o 

Forestry sciences 

Computer and information 

o 

Computer/information 

o 

Computer systems analysis 

sciences For Computer 


sciences, general 

o 

Information services/systems 

Engineering, see 

o 

Computer programming 

o 

OTHER computer/information sciences 

ENGINEERING, next page 

o 

Computer science 



Mathematics and statistics 

o 

Applied mathematics 

o 

Statistics 


o 

Mathematics, general 

o 

OTHER mathematics 


o 

Operations research 



Chemistry, except biochem 

o 

Chemistry except biochemistry (biochemistry, see Biological sciences) 

Earth, atmospheric, and 

o 

Atmospheric 

o 

Geological sciences, other 

ocean sciences 


sciences/meteorology 

o 

Oceanography 


o 

Earth sciences 

o 

OTHER physical sciences 


o 

Geology 



Physics Biophysics, see 

O Astronomy/astrophysics 

o 

Physics 

Biological Sciences 





Psychology 

O 

Clinical psychology 

o 

General psychology 


o 

Counseling 

o 

Industrial/organizational psychology 


o 

Educational psychology 

o 

Social psychology 


o 

Experimental psychology 

o 

OTHER psychology 


SUBMIT 

©Q BACK TO 4 BROAD CATEGORIES 
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Social Sciences and Health/Medical fields 


For Science/Math Education select the primary scientific or mathematical field (click on 
or see below "OTHER Social Sciences: Education, General" 

To view other fields (Science, Engineering), please select Go back to broad tatggwtel to 

change your selection. 


Economics 

O Agricultural economics O Economics 

Political and related 
sciences 

O International relations O Public policy studies 

O Political science/government 

Sociology/Anthropology 

O Anthropology/archaeology O Sociology 

O Criminology 

OTHER social sciences 

O Area/ethnic studies O Linguistics 

O Education, general O Philosophy of science 

O Geography O OTHER social sciences 

O History of science 

Health 

O Audiology/speech pathology O Nursing (4 years or longer program) 

O Health services administration O Pharmacy 

O Health/medical assistants O Physical therapy and other 

O Health/medical technologies rehabilitation/therapeutic services 

O Medical preparatory programs (e.g., O Public health (Including environmental 

pre-dentistry, pre-medical, health/epidemiology) 

pre- veterinary) O OTHER health/medical sciences 

O Medicine (e.g., general, internal, orthopedic, 

surgical, dentistry, optometry, osteopathic, 
podiatry, veterinary) 


SUBMIT 

©O BACK TO 4 BROADER CATEGORIES 
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Engineering fields 

For Science, Math or Engineering Education select the primary scientific or 
mathematical field (click on Sctenm or or select SendiaH Scteneas or HftaWh/MwteaJ and 
see "OTHER Social Sciences: Education, General" 

To view other fields (Science, Social Science, Health/Medical), please select Go back to the 4 
bwadtategwtes to change your selection. 


Biochemical engineering, see Bioengineering/biomedical under OTHER Engineering 

Chemical engineering 

O 

Chemical engineering 



Civil/architectural eng. 

o 

Architectural engineering 

O 

Civil engineering 

Electrical/computer 

o 

Computer/systems engineering 

O 

Electrical/electronics/communications 

engineering 




engineering 

Industrial engineering 

o 

Industrial/manufacturing engineering 



Mechanical engineering 

o 

Mechanical engineering 



OTHER engineering 

o 

Aerospace/aeronautical/ 

o 

Geophysical/geological engineering 



astronautical engineering 

o 

Materials engineering, including 


o 

Agricultural engineering 


ceramics/textiles 


o 

Bioengineering/biomedical 

o 

Metallurgical engineering 



engineering 

o 

Mining/minerals engineering 


o 

Engineering, general 

o 

Naval architecture/marine engineering 


o 

Engineering sciences/ 

o 

Nuclear engineering 



mechanics/physics 

o 

Petroleum engineering 


o 

Environmental engineering 

o 

OTHER engineering 


SUBMIT 

©Q HACK TO 4 BROAD CATEGORIES 
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MODULE B: THE IRFP APPLICATION PROCESS 


Bl. Why did you apply to IRFP? Check all that apply. 

□ To conduct research with a specific person or at a specific institution 

□ To collaborate with a foreign scientist 

□ To understand what research in my field was like outside the US 

□ To access resources (e.g. samples, equipment) for research that I could not find in the U.S. 

□ To enhance my skills or knowledge as a researcher 

□ To enhance my resume as a future job candidate 

□ To learn about the culture, history, and/or geography of another country 

□ To learn another language 

□ To travel outside of the United States 

□ For family reasons (e.g., to accompany a spouse traveling abroad) 

□ Other (please specify): [textbox, 300] 


B2. For your application to the IRFP program, why did you select your IRFP country? Check all that 
apply. 

□ The host researcher was conducting research relevant to my own interests 

□ The host institution had equipment or resources helpful for my proposed research 

□ A faculty advisor or mentor recommended this particular country, the particular host institution 

or host researcher in this country 

□ I have professional ties with someone who is from this country 

□ The host researcher had visited my department or university 

□ There were particular places in this country (aside from the host institution) that I wanted to 

visit or see 

□ I had studied this country's culture, history, politics, geography, etc. before applying 

□ I was familiar with the primary language(s) spoken there 

□ Other reason (please specify): [textbox, 300] 


B3. Did the notification of your application status allow you sufficient time to make the necessary 
arrangements for your visit or to make alternative plans if you did not receive the award? Check one 
only. 

□ Yes 

□ No 
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B4. How did your primary graduate advisor view your decision to apply for an IRFP? Check one only. 

□ I do not know how my advisor viewed my decision to apply 

□ I did not have an advisor when I most recently applied 

□ My advisor opposed my decision to apply 

□ My advisor was indifferent to my decision to apply 

□ My advisor supported my decision to apply 

□ My advisor encourage me to apply to the IRFP program 


B5. What types of mentoring or guidance did you receive from your graduate advisor or other 
colleagues during the preparation of the IRFP application? Check all that apply. 

□ Suggested a host institution 

□ Recommended me to a colleague at the host institution 

□ Provided feedback on my project proposal 

□ Discussed with me cultural and language aspects of the host country 

□ Other -specify: [textbox, 300] 

□ None 
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MODULE C: ABOUT YOU AT THE TIME OF YOUR APPLICATION 

Items in this section ask for information about the year in which you applied for an IRFP Fellowship. 

Cl. At the time you applied for an IRFP Fellowship ([year_of_application]) were you employed in a 
tenure-track faculty position? Check one only 

□ Yes 

□ No 


C2a. At the time you applied for IRFP ([year_of_application]), what was the highest degree you had 
completed ? You may have applied before you officially received your doctorate, for example. 
Check one only: 

□ Bachelor's degree (BS, BA) 

□ Master's degree (MA, MS, MBA, etc.) 

□ Doctoral degree (PhD, EdD, MD, joint MD/PhD, JD, PsyD, ScD) 

□ Other degree- specify: [textbox, 50] no entry req'd in textbox 


C2b. At the time you applied for IRFP ([year_of_application]), had you received or were you pursuing a 
graduate degree from an institution outside the United States? Check one only 

□ Yes 

□ No 

C3a. While you were an undergraduate, did you participate in a study abroad program or did you spend 
a semester (or more) pursuing your education at a college/university outside the United States? 
Check one only 

□ Yes 

□ No 


C3b. While you were a graduate student did you participate in a study abroad program or did you spend 
a semester (or more) pursuing your graduate research or education outside the United States? 
Check one only 

□ Yes 

□ No 
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C4. At the time you applied for IRFP ([year_of_application]), had you done any of the following? Check 
all that apply 

□ Attended elementary or secondary school in another country (outside the U.S.) 

□ Lived outside the U.S. for six months or longer 

□ Collaborated on research with someone based in another country 

□ Published research with someone based in another country 

□ Attended or presented scholarly work at a research conference in another country 

□ Participated on a research team with a scientist who was visiting my group from a foreign 
institution 

□ Worked with a colleague who had completed an IRFP Fellowship 

□ None of the above 

C5a. At the time you applied for an IRFP fellowship ([year_of_application]), were you already working 
at the foreign institution that you proposed as your IRFP host? Check one only 

□ Yes 

□ No 

C5b. At the time you applied for IRFP ([year_of_application]), were you already collaborating with the 
host scientist(s) or someone else at the host institution(s) that you proposed in your application? 
That is, even if you were not physically working at the host institution, were you already 
collaborating with someone at that institution ? Check one only 

□ Yes, with my proposed host scientist 

□ Yes, with someone else at my proposed host institution (not the host scientist) 

□ No 

C6. At the time you applied for an IRFP Fellowship ([year_of_application]) were you aware of any 

relationship between your U.S. -based institution (graduate, postdoctoral, or other) and that of the 
host institution to which you applied? Check all that apply: 

□ Yes, a student, postdoc or faculty member in my department had collaborated with 
someone at the host institution 

□ Yes, my department or university had an existing collaboration or student/faculty exchange 
program with the host institution 

□ Other students/postdocs/faculty from my university or department had been to the host 
institution 

□ Yes, there was another type of relationship. Please specify: [textbox, 300] no entry req'd in 
textbox 

□ No, I was not aware of any such relationship 
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C7. At the time you applied for the IRFP ([year_of_application]), how many of the following had you 
authored/co-authored, edited/co-edited, developed/co-developed? Include works in press but do not 
include works "under review" or "in preparation." Include works published in electronic or printed format 
but do not count the same work more than once if it is available in multiple formats. 


Total number 
published/in- 
press 

Number completed 
with a foreign 
collaborator 




Peer-reviewed journal articles 



Peer-reviewed conference publications (e.g. 
abstracts, conference papers, posters) 



Patents, registered or pending 



Book chapter(s) (e.g., in edited volumes) 


C8. Have you ever received a nationally competitive fellowship to support your graduate studies? Do 
not include support you received directly from your institution or support from a faculty member's 
grant funding. 

Nationally-competitive fellowships are unrestricted fellowships granted by a federal agency, 
private foundation, or similar organization directly to an individual graduate student (or graduate 
school applicant) for use at any graduate institution of his/her choosing. Check one only 

□ Yes 

□ No 

FELLOW, GO TO ITEM C9. 

UNFUNDED, GO TO ITEM Module D (Item Dla) 

C9. As of October 1, 2010, had you completed your IRFP fellowship (including any U.S. -based "re-entry 
period")? Check one only 

□ Yes Go to Dla 

□ No -> R will receive items in Module E and Module G; Do not Present Modules D, F. 
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MODULE D: PROFESSIONAL HISTORY 


In this section, we ask about various professional experiences you've had up to October 1, 2010. We 
use a standard date so that all survey participants think about the same period of time when 
answering these questions. 

Dla. Since receiving your first doctoral degree how many postdoctoral appointments have you held? 
Please include any postdocs you held through October 1, 2010. Please include IRFP if you 

received an award. 

Enter Number of postdocs [textbox 2] or if None, enter 0. 

If 0, skip to Die, else go to Dlb. 

Dlb. [all applicants]: Since receiving your first doctoral degree, how many postdoctoral appointments 
have you held at institution(s) outside the United States? Please include any postdocs you held 
through October 1, 2010. Please include IRFP if you received this fellowship. Check one only 
□ 0 
□ 1 
□ 2 

□ 3 or more 


Die. During the week of October 1, 2010 were you working for pay or profit? Work includes being self- 
employed, on a postdoctoral appointment, or on any type of paid or unpaid leave, including vacation. 
Check one only 

□ Yes, in a postdoctoral position -> GO TO D4 

□ Yes, in another type of position (i.e., not a postdoctoral position) GO TO D2 

□ No, I was not working then -> GO TO D5 


D2. Was your primary employer during the week of October 1, 2010 an educational institution? Check 
one only 

□ Yes GO TO D2a 

□ No GOTOD3 

D2a. Was the educational institution where you worked a . . . Check one only 

□ Preschool, elementary, middle, or secondary school or system -> GO TO D4 

□ Two-year college, community college or technical institute 

□ Four-year college or university,* other than a medical school 

□ Medical school (including university-affiliated hospital or medical center) 

□ University-affiliated research institute 

□ Other educational institution [please specify]: [textbox, 150] 

*Four-year college/university includes doctoral-granting and non-doctoral-granting 
institutions. 
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D2b. During the week of October 1, 2010 what type of academic position did you hold at this 
institution? Mark Yes or No for each item. 


Yes 

No 


□ 

□ 

President, Provost, or Chancellor (any level) 

□ 

□ 

Dean (any level), department head or chair 

□ 

□ 

Research faculty, scientist, associate or fellow 

□ 

□ 

Teaching faculty 

□ 

□ 

Adjunct faculty 

□ 


Other (please specify: [ textbox, 75]) 


D2c. During the week of October 1, 2010 what was your faculty rank? Check one only 

□ Not applicable: no ranks designated at this institution 

□ Not applicable: no ranks designated for my position 

□ Professor/Full Professor 

□ Associate Professor 

□ Assistant Professor 

□ Instructor 

□ Lecturer 

□ Other: [textbox, 75] 

D2d. What was your tenure status? Check one only 

□ Not applicable: no tenure system at this institution 

□ Not applicable: no tenure system for my position 

□ Tenured 

□ On tenure track but not tenured 

□ Not on tenure track 

-» after D2d, SKIP to D4 


D3 only for respondents where D2 = NO (primary employer during the week of October 1, 2010 was 
not an educational institution) 

D3. Which of the following best describes your primary employer during the week of October 1, 2010? 
Check one only 

□ SELF-EMPLOYED or a BUSINESS OWNER (non-incorporated or incorporated business, 
professional practice, or farm) 

□ PRIVATE SECTOR (for-profit or non-profit, including tax-exempt and charitable 

organizations) 

□ Local GOVERNMENT (city, county, school district) 

□ State GOVERNMENT 

□ U.S. federal GOVERNMENT 

□ U.S. MILITARY service, activity duty or Commissioned Corps (e.g., USPFIS, NOAA) 

□ OTFIER type of employer: Please specify: [textbox, 300] 
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D4. In performing the primary job you held during the week of October 1, 2010, did you work with 
individuals located in countries other than the US? If you were a postdoctoral fellow during this 
period, please consider the postdoctoral fellowship to be your primary job. Check one only 

□ Yes ->GOTOD4a 

□ No GO TO D5 


D4a. Did your work with individuals in countries other than the US involve. . . 
Mark Yes or No for each item. 

Yes No 


□ 

□ 

Sharing data or information? 

□ 

□ 

Sharing materials, equipment, or facilities? 

□ 

□ 

Preparing a joint publication? 

□ 

□ 

Jointly developing a product, process, or program? 

□ 

□ 

Collaborating on a research project? 

□ 


Other type of work? Specify: [ textbox, 300] 


D4b. For the primary job you held during the week of October 1, 2010 were you employed at a location 
outside the U.S.? If you were based in the United States but travelled internationally for this job, 
you should answer "No." 

□ Yes 

□ No 


D5. Between [year of IRFP application] and October 1, 2010, did you receive any grants (as Principal 
Investigator or co-Principal Investigator), prestigious awards or honors based on your research? If 
you were an IRFP fellow, do not count the IRFP Fellowship itself. Mark one answer in each row. 


Not applicable for my 
position (e.g., not 
eligible for 
grants/awards) 

Yes 

No 


□ 

□ 

□ 

Grant(s) as Principal Investigator 

□ 

□ 

□ 

Grant(s) as co-Principal Investigator 

□ 

□ 

□ 

Prestigious award(s)/honor(s) 


If any row in D5 = Yes: -> Former IRFP Fellows, GO TO D5a; Unfunded applicants, SKIP to D5b 
If ALL rows in D5 = No or Not applicable -> GO TO D6. 

D5a. [former IRFP Fellows only] Were any of these grants or awards based on research conducted 
or advanced during your IRFP fellowship? Check one only 

□ Yes 

□ No 

□ Not sure 
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D5b. What was the name of the most prestigious grant, award or honor for research you have received 
and who did it come from? 

(i) Name of award: [textbox, 300] 

(ii) The award was from: Check one: 

□ An international organization or foreign government 

□ A U.S. government agency 

□ A US-based professional association 

□ A private foundation in the U.S. 

□ An institution where I was employed or my doctoral institution 

□ None of the above 

(iii) Please type the full name of the awarding agency or organization: [textbox, 300] 


D6. Between [year of IRFP application] and October 1, 2010, how many of the following works had you 
published or produced (on your own or with others)? 

Include works "in press" but do not include works "under review" or "in preparation." Include 
works published in electronic or printed format but do not count the same work more than once if 
it is available in multiple formats. 


Make your best approximation if you do not know the exact number. 

The first row should not appear on-screen and the column "IRFP Fellows only" should appear 
only for FELLOWS not for UNFUNDED APPLICANTS 


Do not display this row onscreen 

All Respondents 

All respondents 

IRFP Fellows only: 


Total 

(if none, enter 
"0") 

How many of these 
from collaboration 
with foreign 
colleague(s)? 

How many of these 
from collaboration 
with your IRFP 
host? 

Peer-reviewed journal articles 




Peer-reviewed conference 
publications (e.g. abstracts, 
conference papers, posters) 




Patents, registered or pending 




Book chapter(s) (e.g., in edited 
volumes) 





IF (year_of _application = 2008) OR (year_of_application = 2009) THEN 


IF (AWDSTATUS =1 AND C9 = YES) THEN DO 
after item D6, SKIP TO ITEM D9; and 
after item D9, SKIP TO ITEM El 
Else IF AWDSTATUS=2 THEN after item D6, SKIP TO ITEM G1 
ELSE skip patterns unchanged. 
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For IRFP applicants who applied for the 2008 or 2009 year, survey items D7, D8, and DIO would not 
make sense. These items include the variable ''[Year of application +2]" which would have equaled 
2010 or 2011. The resulting items would not have made sense. For example, consider Item D7: 

D7. Between [year of IRFP application + 2] and October 1, 2010, have you ever worked in another 
country? ("Work" refers here to employment for pay or profit and includes postdoctoral 
appointments.) 

For IRFP Applicants for the 2008 cohort: 

D7. Between 2010 and October 1, 2010, have you ever worked in . . . 

For IRFP Applicants for the 2008 cohort: 

D7. Between 2011 and October 1, 2010, have you ever worked in . . . 


As a result, after Item D6, the following code was used to SKIP such respondents out of these items: 

IF (year_of _application = 2008) OR (year_of_application = 2009) THEN 
IF (AWDSTATUS =1 AND C9 = YES) THEN DO 
after item D6, SKIP TO ITEM D9; and 
after item D9, SKIP TO ITEM El 
Else IF AWDSTATUS=2 THEN after item D6, SKIP TO ITEM G1 
ELSE skip patterns unchanged. 
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D7. Between [year of IRFP application + 2] and October 1, 2010, have you ever worked in another 
country? ("Work" refers here to employment for pay or profit and includes postdoctoral 
appointments.) Check one only. 

□ Yes GO TO D7a 

□ No -» GOTOD8 

D7a. For how many years (if less than 1 year, how many months) did you work in another 
country (or countries)? 

I worked in another country/countries for: Check one only. 

□ Less than 1 year ->Go to D7a_(i) 

□ 1 year or longer ->Goto D7a_(ii) 

D7a_(i). Enter number of months total: [textbox, 2] -> Go to D8 
D7a_(ii). Enter number of years total: [textbox, 2] -> Go to D8 

D7a(i) and D7a(ii) appear on-screen together with D7a. 


D8. Between [year of IRFP application + 2] and October 1, 2010, did you mentor any individuals from 
the United States who conducted research in another country? 

Check all that apply 

□ I mentored undergraduate students who conducted research abroad 

□ I mentored graduate students who conducted research abroad 

□ I mentored postdocs who conducted research abroad 

□ I mentored faculty who conducted research abroad 

□ I mentored research scientists who conducted research abroad 

□ I mentored other individuals who who conducted research abroad 

□ I did not mentor any individuals who conducted research abroad 

Former IRFP Fellows: Go to D9 

Unfunded applicants: Go to DIO 

(Fellows only) : 

In this next question, we're interested in any effects that your participation in IRFP may have had on 
people or institutions you've worked with in the U.S. since the end of your IRFP fellowship. 

D9. Since the end of your IRFP fellowship, have you done any of the following? 



Yes 

No 

Not 

applicable 

1 have taught my colleagues, students, or peers research 
methods that 1 learned during my IRFP fellowship 

□ 

□ 

□ 

1 have shared with my colleagues resources (e.g., data, 
samples, materials) or tools (e.g., algorithms, software, 
instruments) that 1 collected developed during my IRFP 
fellowship 

□ 

□ 

□ 


Continue to DIO 
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(IRFP Fellows and Unfunded applicants): 

DIO. Between [year of IRFP application +2] and October 1, 2010, have you done any of the following: 



Yes 

No 

Not 

applicable 

1 have established a program to foster international 
collaborations 

□ 

□ 

□ 

1 have hosted researchers or professional colleagues from 
another country at my institution 

□ 

□ 

□ 

1 have led a delegation of colleagues to visit a research 
laboratory, university, or business in another country 

□ 

□ 

□ 

1 have established or served in a leadership role for an 
international association for professionals in my line of 
work 

□ 

□ 

□ 


Unfunded applicants: Go to Module G (DEMOGRAPHICS) 
Fellows: Go to Module E (THE IRFP FELLOWSHIP EXPERIENCE) 
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MODULE E: THE IRFP FELLOWSHIP EXPERIENCE 

In this section, we'd like to ask about experiences you had during your IRFP Fellowship. 

El. In preparation for your IRFP fellowship, did you study a language spoken in your host country? 

Check all that apply. 

□ No, I was already familiar with the language spoken in the host country 

□ No, I did not study a language spoken in the host country 

□ Yes, I did some self-guided language study (i.e., individual study using books or computer-based 
instructional software, such as Rosetta Stone ) 

□ Yes, I studied with a conversation partner or a tutor who was familiar with a language spoken in 
the host country 

□ Yes, I enrolled in a formal language training course led by an instructor (either an online course 
or "live" course) 


E2. During your IRFP fellowship, did you attend or participate in any of the following types of cultural or 
leisure activities in your host country? Check all that apply 

□ Sightseeing 

□ Museums 

□ Festivals, holiday or religious ceremonies 

□ Outdoor activities to explore the landscape, geography 

□ Sporting events 

□ Non-scientific lectures or presentations 

□ Other (please specify): 

□ I did not participate in any cultural activities 

E3. During your IRFP fellowship, did you attend or participate in any of the following types of 
professional activities? Check all that apply 

□ Visit(s) to educational or research institutions other than my host institution 

□ Visit(s) to businesses/industrial laboratories 

□ Language courses or language study 

□ Lectures, colloquia, seminars in my field 

□ Attended professional conferences in my host country 

□ I gave a talk or presentation to researchers from my host country 

□ Networking with colleagues from institutions other than my host institution 

□ Other (please specify): 

□ I did not attend or participate in any professional activities 

E3a. Please describe one of the most memorable activities or events you experienced in your host 
country: 
textbox, 2500 
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E4. Who was primarily involved in the following activities related to the IRFP project(s) on which you 
worked? Mark one answer in each row. 



Mostly me 
independently 

Mostly the 
host or 
members of 
his/her group 
without me 

Me and the 
host or host's 
research 
group 
together 
about equally 

Not applicable 

Developing the ideas, hypotheses, 
broad framework, or vision for the 
research project 

□ 

□ 

□ 

□ 

Researching literature or research base 
relevant to the project 

□ 

□ 

□ 

□ 

Keeping records, tracking supplies, 
resources 

□ 

□ 

□ 

□ 

Developing instrumentation, software, 
equipment, or data collection 
processes 

□ 

□ 

□ 

□ 

Collecting data or carrying out 
simulations 

□ 

□ 

□ 

□ 

Analyzing data or observations 

□ 

□ 

□ 

□ 

Interpreting results 

□ 

□ 

□ 

□ 

Planning or developing follow-up work 
based on results 

□ 

□ 

□ 

□ 

Written, oral dissemination of results 
(publications, presentations) 

□ 

□ 

□ 

□ 
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E5. Did you experience any of the following difficulties during your fellowship? Check all that apply. 

□ Inadequate access to space, facilities, equipment, computers, resources/supplies 

□ My role on the project was less than that merited by my skills/knowledge 

□ Not enough guidance from host/host's research group 

□ I was asked to do work that was someone else's responsibility 

□ Not given credit for my contributions to advancing a project 

□ Communication or language difficulties 

□ Logistical difficulties (e.g., with transportation, navigating bureaucracy, etc. ) 

□ Legal or medical difficulties in my host country 

□ I felt that my ideas were not treated with respect 

□ Encountered barriers or discomfort based on my gender 

□ Encountered barriers or discomfort based on my race/ethnicity 

□ Encountered barriers or discomfort based on my cultural or religious background 

□ Encountered barriers or discomfort based on a disability 

□ Other (please specify): [textbox, 300] 

□ None 


E6. During your IRFP fellowship, who provided direct supervision while you were conducting your work? 
Check all that apply. 

□ The host researcher him/herself 

□ Another staff scientist 

□ A junior faculty member or post-doctoral fellow 

□ Another graduate student(s) 

□ A laboratory technician or other employee/worker 

□ No one, I was not supervised by anyone 
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E7. Please indicate how satisfied you were with various aspects of your IRFP experience. Mark one 
answer in each row. 


Accommodations & Logistics 

Very 

dissatisfied 

Somewhat 

dissatisfied 

Somewhat 

satisfied 

Very 

satisfied 

Lodging 

□ 

□ 

□ 

□ 

Fellowship support amount 

□ 

□ 

□ 

□ 

Fellowship duration 

□ 

□ 

□ 

□ 

Fellowship timing with respect to your career 
goals 

□ 

□ 

□ 

□ 

Research/laboratory facilities at the host location 

□ 

□ 

□ 

□ 

Access to the internet 

□ 

□ 

□ 

□ 






Your host 

Very 

dissatisfied 

Somewhat 

dissatisfied 

Somewhat 

satisfied 

Very 

satisfied 

Frequency of meetings with your host 

□ 

□ 

□ 

□ 

Match between yours and your host's research 
interests 

□ 

□ 

□ 

□ 

Host's expertise in his/her field 

□ 

□ 

□ 

□ 

The level of the host's intellectual contribution to 
your joint research project 

□ 

□ 

□ 

□ 

Host's efforts to help you meet other researchers 

□ 

□ 

□ 

□ 

Host's inclusion of you in research 
group/laboratory, meaningful collaboration 

□ 

□ 

□ 

□ 

Guidance or mentoring provided by host 

□ 

□ 

□ 

□ 

Your experiences 

Very 

dissatisfied 

Somewhat 

dissatisfied 

Somewhat 

satisfied 

Very 

satisfied 

Opportunities to experience and learn about your 
host country, its culture, history, etc 

□ 

□ 

□ 

□ 

The quality of research you were able to conduct 

□ 

□ 

□ 

□ 

Professional connections you made during the 
fellowship 

□ 

□ 

□ 

□ 


E7a. If you would like to elaborate or comment further on areas of satisfaction or dissatisfaction with 
your IRFP experience, please do so: 

textbox, 2500 
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MODULE F: AFTER YOUR IRFP FELLOWSHIP 


FI. Since the conclusion of your IRFP fellowship and October 1, 2010, have you collaborated or 
communicated with your host scientist? Check one only 

□ I have collaborated on a research project -> GO TO Fla 

□ I have communicated with my host but haven't collaborated 
further on research -> GO TO Flc 

□ I have not communicated with my host -> GO TO Flc 

Fla. What was the extent of your collaboration? Check all that apply 

□ I have a position in the host's group 

□ I have a position at the host's institution 

□ We exchanged ideas, data, ideas, research results, or tools 

□ We co-authored papers 

□ We co-advised students 

□ We visited each other at our institutions 

□ Other - specify [textbox, 300] 

Fib. How recently has the latest collaboration with your host occurred? Check one only 

□ Within the past 6 months GO TO F2 

□ Within the past 12 months -> GO TO F2 

□ 1-2 years ago GO TO F2 

□ 3 or more years ago GO TO F2 

Flc. Why do you no longer collaborate with your former host? Check all that apply 

□ Our research interests diverged 

□ One or both of us lacked funding needed to maintain collaboration 

□ Language differences have hindered further collaboration 

□ Political or cultural differences have hindered further collaboration 

□ Geographic distance has hindered further collaboration 

□ I did not think that further collaboration would be beneficial for me 

□ My host did not actively pursue or maintain further collaboration with me 

□ One (or both) of us is too busy with other projects 

□ Other - specify [textbox, 300] 
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F2. Did participating in IRFP make you qualified for a broader range of opportunities after the fellowship 
ended? Check one only 

□ Yes, IRFP broadened my career options-} Go to F2a 

□ No, IRFP did not broaden my career options-} Go to F2b 

□ I am not sure -} Go to F3 

F2a. Describe how IRFP broadened your career options: 
textbox, 2500 


SKIP TO F3 

F2b. Did IRFP constrain your career options? Check one only 

□ Yes -} Go to F2c 

□ No-} Go to F3 

F2c. Describe how IRFP constrained your career options: 
textbox, 2500 


F3. Which of the following professional benefits occurred as a result of your participation in IRFP? 

Check all that apply 

□ My work at the host institution resulted in a substantial advancement in my research 

□ My work at the host institution opened up new areas of investigation 

□ I became familiar with scientific enterprise of the host country 

□ I became committed to international research collaboration 

□ I made valuable connections to researchers in the host country 

□ My career goals changed from an academic career to a non-academic career 

□ My career goals changed from a non-academic to an academic career 

□ I decided to pursue research in a different discipline than the one I was most familiar with when 
I began my IRFP fellowship 

□ IRFP participation made me more competitive for jobs I was interested in 

□ IRFP participation made me consider professional opportunities I would not have considered in 
the past 

□ None of the above 

F3a. What did your IRFP host scientist do to assist you in obtaining employment after your fellowship? 
Check all that apply. 

□ Helped me network with potential employers 

□ Recommended me to others directly, by telephone or in-person 

□ Reviewed my CV/resume and/or other application materials 

□ Wrote letters of reference on my behalf 

□ Provided financial assistance for travel to conference, other professional networking 
opportunities 

□ Other (please specify): [textbox 300] 

□ Host offered me no assistance finding post-fellowship employment 
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F4. Which of the following additional benefits occurred as a result of your participation in IRFP? Check 
all that apply 

□ Research methods or ideas that I learned benefited others in my institution 

□ Samples that I collected or tools that I developed benefited others in my institution 

□ My peers became interested in international collaboration 

□ Others in my research group (in the U.S.) began an international research collaboration 

□ Researchers that I met during my fellowship joined my research group in the U.S. 

□ Other - specify [textbox, 300] 

□ None of the above 

F5. Which of the following personal benefits occur as a result of your participation? 

Check all that apply. 

□ I became comfortable with the traditions and culture of the host country 

□ I made personal connections in the host country 

□ I gained proficiency in another language 

□ I decided to live outside the United States (i.e., at least 6 months) 

□ None of the above 


F6. Which of the following are true? Check Yes or No for each 


Yes 

No 


□ 

□ 

Obtaining full-time employment after the conclusion of my IRFP 
fellowship was more difficult than 1 expected 

□ 

□ 

1 lost an important career or educational opportunity by 
participating in IRFP 

□ 

□ 

1 am more skeptical about international collaboration than before 
IRFP 


F7. What was the most positive aspect of your IRFP experience? 


Textbox 2500 


F8. What was the most challenging aspect of your IRFP experience? 


Textbox 2500 
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F9. Would you recommend IRFP to a colleague? Check one only 

□ Yes 

□ No 

F9a. Why or why not? 
textbox, 2500 


F10. Would you recommend your IRFP host scientist to a colleague seeking an international 
collaboration? Check one only 

□ Yes 

□ No 


Fll. Would you recommend your host country to a colleague? Check one only 

□ Yes 

□ No 


F12. What would you change about the IRFP program? 
Textbox 2500 


F13. What was the most important contribution of IRFP participation to your career? 
Textbox 2500 
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MODULE G: DEMOGRAPHIC INFORMATION 


Gl. What is your gender? 

□ Male 

□ Female 

G2. What is your ethnicity? Mark one only. 

□ Hispanic or Latino 

□ Not Hispanic or Latino 

G3. What is your race? Check one or more. 

□ American Indian or Alaska native 

□ Asian 

□ Black or African American 


Go to G5 
-> Go to G4a 

-> Go to G4a 
-> Go to G4a 


□ Native Hawaiian or other Pacific Islander 

□ White 

G4. What is your citizenship status? 

□ United States citizen since birth 

□ United States citizen, naturalized 

□ Non-U. S. citizen with a permanent 
U.S. Resident Visa ("Green Card") 

□ Non-U. S. citizen with a temporary U.S. Visa 


G4a. How old were you when you began living in the United States? (age in years): [textbox, 2] 

The next question is designed to help us better understand the career paths of individuals with 
different physical disabilities: 

G5. What is the USUAL degree of difficulty you have Unable 


with: Mark one answer for each item. None Slight Moderate Severe to do 


1 SEEING words or letters in ordinary newsprint (with 
glasses/contact lenses, if you usually wear them) 

□ i 

□2 

□3 

□4 

□5 

2 HEARING what is normally said in conversation with 
another person (with hearing aid, if you usually wear 
one) 

□i 

□2 

□3 

□4 

□5 

3 WALKING without human or mechanical assistance or 
using stairs 

□ i 

□2 

□3 

□4 

□5 

4 LIFTING or carrying something as heavy as 10 pounds, 
such as a bag of groceries 

□ i 

□2 

□3 

□4 

□5 


If all 4 rows in G5 are marked "None," Go to Thank you screen. Else go to G5a. 

G5a. What is the earliest age at which you first began experiencing any difficulties in any of these 
areas? 


□ Since birth; or Enter age in years: 
valid data range is 0 to 99yrs 


SUBMIT SURVEY 
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Thank you screen: After hitting SUBMIT, respondent sees this screen. 


IRFP applicant survey 


Conclusion 

Thank you for completing this survey. If you have any general comments about the survey, please write 
them below. 

Textbox, 2500 


If you have any questions about this survey or the study, please contact Dr. Alina Martinez, Study 
Director, Abt Associates, at (866) 421-6223 (toll-free within the US) or email her at 
IRFP_survey@abtassoc.com. You may also contact John Tsapogas, who is overseeing this study at NSF, 
with any questions or comments: jtsapoga@nsf.gov. 

Thank you for your assistance. We greatly appreciate your time and consideration. 
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CONFIRM FELLOWSHIP SCREEN 

Programming note: From these screens respondent should be able to go back to the previous item that 
triggered this screen. For example, if they accidentally selected a response on A2.1 or A2.2 that they did not 
intend, they need the option to return and fix it. 

Before continuing, we'd like to note that NSF records may have an error. For example, the records 
may have confused the International Research Fellowship Program (IRFP) and the NSF-NATO 
Postdoctoral Science and Engineering Fellowship program. The NSF-NATO postdoc differs from the 
IRFP award: 

■ NSF-NATO postdoctoral fellowships provided up to 12 months of funding for research in one of 
the NATO member countries or NATO partner nations: 

Albania, Armenia, Azerbaijan Belgium, Belarus, Bulgaria, Canada, Czech Republic, Denmark, 
Estonia, France, Georgia, Germany, Greece, Hungary, Iceland, Italy, Kazakhstan, Kyrgyz Republic, 
Latvia, Lithuania, Luxembourg, Moldova, the Netherlands, Norway, Poland, Portugal, Romania, 
Russian Federation, Slovak Republic, Slovenia, Spain, Tajikistan, Turkey, Turkmenistan, Ukraine, 
the United Kingdom, Uzbekistan or the former Yugoslav Republic of Macedonia; 

■ IRFP fellowships provided 3 to 24 months of funding for research in any country . 

Some survey participants applied separately for both IRFP and NSF-NATO (and/or other postdoctoral 
programs). In a few instances, NSF records may have classified an applicant to the International 
Research Fellowship Program incorrectly. We ask for your forbearance. 

Currently, NSF's records indicate that you (if FELLOW display Section S only; if UNFUNDED APPLICANT 
display Section R only) 
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Section R (unfunded applicants) 

applied for, but did not participate in, a postdoctoral fellowship through NSF's "International 
Research Fellowship Program (IRFP)." You may have participated in a different NSF postdoctoral 
program, OR these records may be mistaken. Please select the response below that is most accurate: 

□ I applied for a postdoctoral fellowship from NSF's IRFP program, but participated in a 
postdoctoral fellowship from a different NSF program (for example, the NSF-NATO program) 

Go To R1 

□ I applied for, but did not participate in, a postdoctoral fellowship from NSF's IRFP program ->Go 
to R1 

□ I applied for, and did participate in, a postdoctoral fellowship from NSF's IRFP program. 

GOTO R2 

□ I never applied to NSF for any internationally-based postdoctoral fellowship. -> Go To EXIT 
SCREEN 

Rl: Because you applied to NSF's "International Research Fellowship Program (IRFP)," we especially 
need you to complete this survey. The survey focuses on your professional achievements, the type 
of work you do currently and the types of collaborations you may have with scientists and engineers 

in the US and other countries. The value of this study to NSF and to the broader scientific 
community depends on the participation of individuals who applied for an IRFP postdoctoral 
fellowship. Please click here to return to the survey: RETURN Go to Item A2. 3a 

A2.3a Did you apply for a [IRFP_Year] IRFP postdoctoral fellowship? 

□ Yes 

□ No, I applied for a different year (Enter four-digit year): [yyyy] Set IRFP_Year = entry and 
FLAG IRFP_YEAR_CHANGE = 1 

AFTER A2.3a, GO TO A3 

R2 (award status change): We apologize for the error, and we thank you for correcting our records. 
We especially need you to complete this survey. This survey focuses on your international experiences, 
professional achievements, the type of work you do currently and the types of collaborations you may 
have with scientists and engineers in the US and other countries. If you would please continue, we 
have updated your status to "former IRFP postdoctoral fellow." Please click here to return to the 
survey. RETURN 

SET AWDSTATUS = 1 AND GOTO Item A2.3b. FLAG RESPONDENT AS 'AWDSTATUS_CHANGE=1' for 
our analysis purposes. 

A2.3b Did your IRFP postdoctoral fellowship begin in [I RFP_Year] ? 

□ Yes 

□ No, it began in (Enter four-digit year): [yyyy] Set IRFP_Year = entry and FLAG 
IRFP_YEAR_CHANGE = 1 

AFTER A2.3b, GO TO A3 
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Section S (awardees) 

participated in a postdoctoral fellowship through NSF's "International Research Fellowship Program 
(IRFP)." You may have participated in a different NSF postdoctoral program, OR these records may be 
mistaken. Please select the response below that is most accurate: 

□ Yes, I participated in a postdoctoral fellowship from NSF's IRFP program -> Go to SI 

□ I participated in an NSF-supported postdoctoral fellowship outside the U.S., but I'm not 
sure what NSF program supported this postdoc. -> Go to SI 

□ I applied for, but did not participate in, a postdoctoral fellowship from NSF's IRFP program 

Go to S2 

□ I never applied for a postdoctoral fellowship from NSF's IRFP program. -> Go to EXIT 
SCREEN 

SI: We especially need you to complete this survey. This survey focuses on your international 
experiences, your professional achievements, the type of work you do currently and the types of 
collaborations you may have with scientists and engineers in the US and other countries. Please click 
here to return to the survey: RETURN Go to Item A2.3b 

A2.3b Did your IRFP postdoctoral fellowship begin in [IRFP_Year]? 

□ Yes 

□ No, it began in (Enter four-digit year): [yyyy] Set IRFP_Year = entry and FLAG 
IRFP_YEAR_CHANGE = 1 

AFTER A2.3b, GO TO A3 

S2 (AWARD STATUS CHANGE) Thank you for correcting our records - we apologize for the error and 
we have updated your status. We especially need you to complete this survey. The value of this 
study depends on the participation of individuals who applied for an IRFP postdoctoral fellowship. 
Please click here to return to the survey. RETURN 

SET AWDSTATUS = 2 AND GOTO Item A2.3a. FLAG RESPONDENT AS 'AWDSTATUS_CHANGE=1' 
for our analysis purposes. 

A2.3a Did you apply for a [IRFP_Year] IRFP postdoctoral fellowship? 

□ Yes 

□ No, I applied for a different year (Enter four-digit year): [yyyy] Set IRFP_Year = entered 
text and FLAG IRFP_YEAR_CHANGE = 1 

AFTER A2.3a, GO TO A3 
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EXIT SCREEN: 

If we have identified the wrong respondent or have erroneous information, display this screen and 
FLAG this respondent for Abt follow-up (weekly basis) 


IRFP applicant survey 


Please accept our apology. 

The information you supplied suggests that you are not eligible to participate in this study or that we 
have reached you in error. We regret any inconvenience to you. If you have any questions about this 
study or you would like to make a comment, please contact one of the following individuals: 

■ Dr. Alina Martinez, Study Director at Abt Associates: IRFP_survey@abtassoc.com, or (866) 421- 
6223 (toll free within the U.S.); 

■ John Tsapogas, Office of International Science and Engineering, NSF: jtsapoga@nsf.gov. 

May we have permission to contact you by telephone in order to clarify your responses here? Entering 
your number does not obligate you to answer any questions. 

□ Yes, you may reach me at . [telephone digit entry]. Go to Best Times 

□ No, please do not contact me. 

Best time(s) to call (check all that apply): 

□ Weekdays (9 to 5pm) 

□ Evenings (5 to 8pm) 

□ Saturday or Sunday 9-5pm 

THIS PAGE SHOULD NOT REQUIRE ANY ENTRY 
THIS PAGE SHOULD HAVE A PREVIOUS ITEM BUTTON 
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Defaults: If Al, A2.1/A2.2, and/or A4a. are blank, no reminder needed but use the default 
below for skip patterns: 

Al. Based on the information that you provided to NSF in your application, your name is [First Name, 
Middle Name/initial, Last Name], Is this correct? 

□ Yes, this is correct. 

[I RFP awardees only] 

A2.1. Based on the information contained in the NSF database, you participated in the National Science 
Foundation's International Research Fellowship Program starting in [Year of application]. Is this 
correct? If you participated in IRFP more than once, please consider your most recent experience 

□ Yes, this is correct. -A GO TO A3 
[Unfunded applicants only] 

A2.2. Based on the information contained in the NSF database, you applied for the National Science 
Foundation's International Research Fellows Program (IRFP) in [year of application], but did not 
participate in a fellowship. Is that correct? 

□ Yes, this is correct. GO TO A3 

A4a. Our records indicate that you received your doctorate in [phdyear]. Is this correct? 

□ Yes, this is correct. -AGOTOA5 


Reminders to display on first time item is left blank: 

A3. Based on the information in the NSF database, you applied to spend your IRFP fellowship in [IRFP 
country]. Is this correct? 

Please select a response before continuing to the next item, [it is ok if this item is still left blank] 

A5. What discipline best characterizes your field of research when you applied for an IRFP fellowship 
(that is, your doctoral field of research)? First, indicate which one of four broader areas in which 
your discipline falls: 

Check one only: 

Please select a response before continuing to the next item, [it is ok if this item is still left blank] 

□ i Sciences (Biological/Life, Chemical, Computer/Information, Environmental, 

Earth/Atmospheric/Oceanographic, Geosciences, 
Mathematical/Statistical, Physics, Psychology) 

□ 2 Social Sciences (Economics, Sociology, Anthropology/Archaelogy, Political Science, 

Geography, Linguistics, other) 


□ 3 Flealth/Medical 
□ 4 Engineering 
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[and it's also ok if no 3-digit code/subcategory is selected within "Sciences" or "Social Sciences" or 
"Health/Medical" or "Engineering" -- no reminder needed] 

Cl. At the time you applied for an IRFP Fellowship ([year_of_application]) were you employed in a 
tenure-track faculty position? Check one only 

No response selected. Please select a response to the best of your recollection, [it is ok if this item is 
still left blank] 

C2a. At the time you applied for IRFP ([year_of_application]), what was the highest degree you had 
completed ? You may have applied before you officially received your doctorate, for example. 

Check one only: 

Please select a response before continuing to the next item, [it is ok if this item is still left blank] 

C9. As of October 1, 2010, had you completed your IRFP fellowship (including any U.S. -based "re-entry 
period")? Check one only 

Please select a response before continuing to the next item, [it is ok if this item is still left blank] 

Die. During the week of October 1, 2010 were you working for pay or profit? Work includes being self- 
employed, on a postdoctoral appointment, or on any type of paid or unpaid leave, including 
vacation. Check one only 

Please select a response before continuing to the next item, [it is ok if this item is still left blank 

D2. Was your primary employer during the week of [October 1, 2010] an educational institution? Check 
one only 

No response selected. If you did not mean to leave this item blank, please check one: 

[if left blank, default is "yes" and go to D2a] 

D4. In performing the primary job you held during the week of [October 1, 2010], did you work with 
individuals located in countries other than the US? Check one only. If you held a postdoctoral 
appointment at this time, please consider that your primary job. 

No response selected. If you did not mean to leave this item blank, please check one: 

[if left blank, default is "no" and go to D5] 

D7. Between [year of IRFP application + 2] and October 1, 2010, have you ever worked in another 
country? ("Work" refers here to employment for pay or profit and includes postdoctoral 
appointments.) Check one only. 

No response selected. If you did not mean to leave this item blank, please check one: [if left 
blank, default is "no" and go to D8] 

□ Yes -> GO TO D7a 

□ No ^ GOTOD8 
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International Research Fellowship Program (IRFP) 
IRFP Host Survey Items 


Host Survey 


1 


IRFP HOST SURVEY 

Programming notes (not to be displayed) will appear in red or brown text throughout. 

Hyperlinked text will apoear tffllfeQSStdl iwdl wdtriijiflifdl in this document but should not be formatted 
this way onscreen unless noted (underlining is fine except for email addresses, but embossing is not). 

General guidelines: 

Each screen should display one question at a time unless otherwise indicated. 

Each screen should display a standard set of Navigation buttons shown below. 



Some items contained placeholders for fields that were programmed to be auto-filled from the sample 
file for each respondent. These fields are enclosed in [brackets] and printed in brown. Textboxes for 
open-ended responses are indicated with [textbox, 150] where the number indicates the length of the 
field (how many characters the respondent may type) 

Sample variables used in Host Survey: 

Fellow name: The first and last name of the IRFP fellow from the sample file 

Host First Name, Host Last name: first and last name of the IRFP HOST from the sample file 

Year of IRFP: The year that the Fellow began his/her fellowship 

Host institution: The name of the institution where the IRFP fellow and host were based. 
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Survey of Host Scientists for former 
NSF-IRFP Postdoctoral Fellows 

A study for the National Science Foundation of the United States 


Welcome and thank for your interest in this study. This survey is being conducted by Abt Associates Inc. and 
our subsidiary, Abt SRBI, for the National Science Foundation (NSF), to learn about the experiences of individuals 
who participated as a host to a postdoctoral fellow supported by NSF's International Research Fellowship 
Program (IRFP). This survey will give NSF information about the international collaborations of U.S. postdoctoral 
fellows and help NSF improve programs intended to foster a globally engaged scientific and engineering 
workforce. You are receiving this survey because you are listed in the IRFP database as a host of a former IRFP 
postdoctoral fellow. We estimate that it will take approximately 15 minutes to complete the survey. 

Confidentiality and Participation 

Participation in the survey is voluntary and nonparticipation will have no impact on you or your institution. You 
may skip questions on the survey or discontinue participation at any time. There are minimal risks associated 
with your participation. We take your privacy very seriously. Your responses to this survey will be protected 
under the U.S. Privacy Act. There is minimal risk of breach of confidentiality, and we have put in place 
procedures to minimize this risk. Reports will never identify you by name, and information from the study will 
only be reported in the aggregate at the program level, combined with about 250 other responses. We will not 
reveal responses that you provide on any question in this survey to any of the fellows that you hosted in your 
institution. Neither will any data that can identify any specific fellow or host be shared with the sponsor of this 
survey (NSF). When we receive your survey we will detach and store separately your name and other identifying 
information that could be used to link you to your survey responses. Survey responses will be stored on a secure 
drive that is only accessible to members on the study team. Only study team web technicians and data analysts 
from Abt Associates and Abt SRBI will see individual responses that can be linked to you. Survey data files will be 
shared with NSF at the end of the study, only after study team members have examined the data to be free of 
any information that could help identify you; this cleaning includes procedures to prevent someone from 
inferring your identity by analyzing non-identifying data. Hence, we encourage you to respond candidly about 
your experiences. Separate from your individual responses to the survey we will provide NSF any updated 
contact information we have found or requested from you. None of this contact information will be linked in 
any way to your survey responses. At the conclusion of the study, Abt Associates and Abt SRBI will destroy all 
records, electronic or otherwise, that link you to your survey responses. 

Questions about the Survey 

If you have questions about the study, please contact the study director, Alina Martinez of Abt Associates Inc. at 
(866) 421-6223 (may incur international telephone charges if initiated outside the U.S) or email her at 
IRFP survey@abtassoc.com . You may also contact the evaluation's program officer at NSF, John Tsapogas 
( jtsapoga@nsf.gov ). If you have questions about your rights as a research participant, you may contact Teresa 
Doksum, the Abt Institutional Review Board Administrator at (877) 520-6835 (may incur international telephone 
charges if initiated outside the U.S) or by email: irb@abtassoc.com . To learn more about this study, please refer to 
thePreqwwtyAsl^CKMStiQns page. 

Consent 

Please click on "Begin" if you agree to participate in this study. peiN] 

This study's IRB approval number is #0494, valid from 8/6/2010 to 8/5/2011. For questions, please contact 
Teresa Doksum, IRB Administrator, Abt Associates, at irb@abtassoc.com . The valid OMB control no. for this 
information collection is 3145-0214. (Expires on 12/31/13) 
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Navigating through the survey: 

As you work through the survey, your responses are automatically saved. You may change a response 
by clicking on the PREVIOUS HEM button. Use the NEXT ITEM button to advance to the next 
question. At any time, you may close your browser if you wish to return and finish at a later time. 
When you re-open the survey you will be able to click on a link that will take you to the question 
where you left off. On each page of the survey, you may click on the 'fAQs w button if you have a 
question during the survey or need information about how to contact the survey administrator. 


When you have completed the survey, please click on theSUBMIT button at the end of the survey. 
You may submit the survey even if there are some questions that you choose not to answer. 


CONTINUE 
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FAQs are Optional screens, displayed only if R clicks on Frequently Asked Questions button 

FREQUENTLY ASKED QUESTIONS 


• What is the International Research Fellowship Program (IRFP)? 

• Why are you doing this study? 

• Why have I been selected to participate in this survey? 

• How did you get my contact information? 

• How long will this survey take to complete? 

• How will you use my comments? 

• Does this study have human subjects review clearance? 

• Who is funding the study? 

• Who are Abt Associates Inc and Abt SRBI? 


What is the International Research Fellowship Program (IRFP)? 

The International Research Fellowship Program (IRFP) is a National Science Foundation 
(NSF) program that provides support for new doctoral-level scientists and engineers to conduct 
postdoctoral research in a country outside the U.S. for a period of 9 to 24 months. 

Why are you doing this study? 

NSF is interested in learning about the experiences of researchers who have hosted one or more 
IRFP fellows from the United States as part of the IRFP program. In particular, NSF would 
like to understand how host researchers perceive the program and the postdoctoral fellows they 
have hosted, and what kinds of benefits and challenges host researchers experienced. The 
information collected in the study will help NSF make improvements to the program, and 
understand how best to support and encourage international collaboration. 

Why have I been selected to participate in this survey? 

You have been selected to participate because we have identified you as having hosted one or 
more U.S. postdoctoral fellows participating in the IRFP program. 

How did you get my contact information? 

We identified you from records maintained by the National Science Foundation on IRFP 
participants and the scientists who hosted them. We then confirmed your contact information 
through an internet search. 

How long will this survey take to complete? 

We estimate that the survey will take about 15 minutes. 

How will you use my comments? 

Responses from all survey respondents will be used to answer questions about the experiences of 
IRFP host researchers with the program and with the guest fellows. We will not reveal responses 
that you provide on any question in this survey to any of the fellows that you hosted in your 
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institution. Neither will any data that can identify any specific fellow or host be shared with the 
sponsor of this survey (NSF). 


Has this study been reviewed and approved by a human subjects review board? 

Yes, the study was approved by Abt Associates’ Institutional Review Board. If you have any 
concerns about your participation in this survey, please contact Teresa Doksum, Institutional 
Review Board Administrator at Abt Associates, at (877) 520-6835 or via email at 
irb @ abtassoc .com. 

Who is funding the study? 

The study has been funded by the National Science Foundation under contract 
GS-10F-0086K. Abt Associates and Abt SRBI will complete the study. 

Who are Abt Associates Inc and Abt SRBI? 

Abt Associates is an independent research firm headquartered in Cambridge, Massachusetts. Abt 
SRBI is a wholly-owned subsidiary of Abt Associates specializing in large-scale data collection 
and public opinion research. NSF has contracted with Abt Associates and Abt SRBI to design 
and implement a survey of scientists who hosted a U.S. postdoctoral fellow funded by NSF’s 
International Research Fellowship Program (IRFP). 
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Survey of Host Scientists for former 
NSF-IRFP Postdoctoral Fellows 


Module A: Verifying Information About You 

We would like to ask you about your experiences with hosting a postdoctoral fellow supported by NSF's 
International Research Fellowship Program. Please respond to the best of your recollection. To begin, 
we'd like to confirm that we've reached the appropriate person. 

Please note: We will not reveal responses that you provide on any question in this survey to any of the 
fellows that you hosted in your institution. Neither will any data that can identify any specific fellow 
or host be shared with the sponsor of this survey (NSF). 

Al. Based on NSF records, you once hosted a postdoctoral fellow from the U.S. named [Fellow name]. 

Is this information correct? Mark one answer 

□ Yes, this is correct. 

□ I hosted this person but his/her correct name is: [Textbox, 75] 

□ No, I did not host the person named above. GO TO EXIT SCREEN 

□ I do not recall. GO TO EXIT SCREEN 

A2. Based on NSF records, your name is [First Name, Last Name], Is this information correct? Mark one 
answer 

□ Yes, my name is correct. 

□ No, my name has changed or is misspelled above. My name is: [Textbox, 75] 

□ No, I am not the person named above. I believe you have reached me by mistake. -> GO TO 

EXIT SCREEN 

A3. Based on NSF records, you hosted [Fellow name] starting in [Year of I RFP] . Is this information 
correct? Mark one answer 

□ Yes, this is correct. 

□ No, this is incorrect. This fellow first began a postdoctoral fellowship with me in (Enter 4-digit 
year): [Textbox, yyyy] 

A4. Based on NSF records, you hosted [Fellow name] at [Host institution]. Is this information correct? 
Mark one answer 

□ Yes, the institution is correct. 

□ No, the institution is incorrect. The correct institution is: [Textbox, 150] 

A5. As part of his/her postdoc, [Fellow name] received funding from NSF's International Research 
Fellowship Program (IRFP). How familiar are you with the IRFP program? Mark one answer 

□ Until now, I had never heard of this particular program 

□ I know that this postdoctoral fellow had received funding from NSF but I was only vaguely aware 
of the IRFP program 

□ I recognize this program by its name but know little else about it 

□ I recognize this program and I'm also familiar with its goals 
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A6. During the week of October 1, 2010, were you working at [Host institution], the institution where 
you worked when you hosted this IRFP Fellow? Mark one answer 

□ Yes 

□ No, I no longer work there. I am now working at (please tell us the name of your current 
employer, university, or company): [Textbox, 150] 


A7. Which of the following characterizes the department (or research unit within an institute) where 
you hosted [Fellow Name]? Check all that apply. 

□ The department (or research unit) where I hosted this person encouraged international 
collaborations 

□ Faculty (or employees) in the department (or research unit) where I hosted this person were 
rewarded for developing international research partnerships 

□ The department (or research unit) where I hosted this person provided financial support to 
faculty (or employees) pursuing international collaborations 

□ The department (or research unit) where I hosted this person promoted (that is, advertised) 
fellowships and other opportunities for researchers in my country to conduct research in 
another country 

□ The department (or research unit) where I hosted this person hosted foreign researchers visiting 
my institution for research-related purposes 

□ None of the above 

□ I do not recall 
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Module B: Deciding to Host an IRFP Postdoctoral Fellow 


The remaining items on this survey refer to the IRFP Fellow named above [Fellow name] and (or) to 
the period of time during which you hosted this individual at your institution (starting in [Year of 
IRFP]). If you have hosted more than one IRFP Postdoctoral Fellow, please answer items on this survey 
with respect to [Fellow name], who you hosted beginning in [Year of IRFP]. 

Bl. How did you first learn about the IRFP program? Check all that apply 

□ From the IRFP postdoctoral fellow I hosted 

□ From the IRFP fellow's PhD (doctoral) advisor 

□ From another faculty member at the IRFP fellow's institution in the United States 

□ From an administrator at the IRFP fellow's institution 

□ From the United States' National Science Foundation (NSF) web site, printed publication, or 
presentation 

□ From a faculty member (or colleague) at my institution 

□ From an administrator at my institution 

□ From a colleague at an institution other than where I work 

□ I do not recall 

□ Other. Please specify: [Textbox, 300] 


B2. Why did you decide to host this IRFP fellow [Fellow Name]? Check all that apply 

□ I was interested in the project proposed by the Fellow 

□ I was interested in establishing or maintaining collaboration with a US researcher 

□ I personally knew, knew of, or previously collaborated with the fellow 

□ I personally knew, knew of, or previously collaborated with the fellow's doctoral advisor 

□ I personally knew, knew of, or previously collaborated with researchers at the fellow's 
institution 

□ I had a positive experience with the IRFP program in the past 

□ I had a positive experience with other U.S. postdoctoral fellows or visiting researchers not 
funded through IRFP 

□ To create an international environment in my research group 

□ To attract students/postdocs to my research 

□ To learn new methodologies, approaches, or tools from the fellow 

□ In my field, individuals trained at U.S. graduate institutions are highly sought-after 

□ My research area is particularly suitable for international collaboration 

□ Other: [Textbox, 300] 
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B3. Did you have any of the following concerns about hosting this IRFP postdoctoral fellow? Check all 
that apply 

□ The fellow's proposed project was especially risky 

□ I was concerned about the fellow's level of commitment to a collaboration with me 

□ I was concerned about the risks of international collaboration in general 

□ I had a negative experience with the IRFP program in the past 

□ I had a negative experience with other U.S. postdoctoral fellows (not funded through IRFP) 

□ I was concerned about the integrating this postdoctoral fellow into my research group 

□ I was concerned that I might not (or my research group might not) benefit from hosting this 
fellow 

□ In my field, individuals trained at U.S. graduate institutions sometimes have gaps in their 
knowledge, skills, or abilities 

□ My research area is not particularly suitable for international collaboration 

□ Other, please specify: [Textbox, 300] 


B4. Before hosting [Fellow Name] starting in [Year of IRFP], had you ever before hosted a postdoctoral 
fellow from the United States ? Mark one answer 

□ Yes, before hosting this person, I had hosted one or more other postdoctoral fellows from the 
U.S. 

□ No, I had never before hosted a postdoctoral fellow from the United States 


B5. Prior to hosting this IRFP postdoctoral fellow (i.e., prior to [Year of IRFP]), had you ever visited the 
United States for educational, research, or other professional purposes? Check all that apply 

□ I was an undergraduate student in the United States 

□ I was a graduate student in the United States 

□ I was a postdoctoral fellow in the United States 

□ I was a visiting scientist in the United States 

□ I was a faculty member in the United States 

□ I came to the United States to attend a conference, a workshop, or a meeting 

□ Other. Please specify: [Textbox, 300] 

□ Before hosting this postdoctoral fellow, I had not visited the United States for any professional 
purposes 

B6. Did you know [Fellow Name] or one (or more) of his/her colleagues before you hosted this IRFP 
postdoctoral fellow in your country? Mark one answer 

□ Yes, I knew the fellow before he (or she) came to my country as a postdoctoral fellow 

□ I did not know the fellow but I knew one (or more) of his/her colleagues 

□ I knew the fellow and one or more of the fellow's colleagues 

□ No, I did not know the fellow or any of his/her colleagues 

B7. Flas [FELLOW NAME]'s IRFP postdoctoral fellowship concluded? 

O Yes, this fellow's IRFP postdoctoral fellowship has ended 
O No, this fellow's IRFP postdoctoral fellowship is still in progress 
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Module C: Your Experiences as Host to an IRFP Postdoctoral Fellow 

IF B7=NO, SHOW TEXT: “For the remaining items in this survey, please respond based on your 
experiences working with this IRFP postdoctoral fellow so far." on separate screen before Cl. 


Cl. Who was primarily involved in the following activities related to the research on which you and the 


IRFP postdoctoral fellow worked 

? Mark one answer in each row. 


Mostly the 
fellow 

independently 

Mostly me 
or members of 
my group 
without the 
fellow 

The fellow and 1 
(and/or members 
of my research 
group) together 
about equally 

Not applicable 
(this activity 
was not part 
of the 
research) 

Developing the ideas, hypotheses, 
broad framework, or vision for the 
research project 

□ 

□ 

□ 

□ 

Researching literature or research 
base relevant to the project 

□ 

□ 

□ 

□ 

Keeping records, tracking supplies, 
resources, 

□ 

□ 

□ 

□ 

Developing instrumentation, 
software, equipment, or data 
collection processes 

□ 

□ 

□ 

□ 

Collecting data or carrying out 
simulations 

□ 

□ 

□ 

□ 

Analyzing data or observations 

□ 

□ 

□ 

□ 

Interpreting results 

□ 

□ 

□ 

□ 

Planning or developing follow-up 
work based on results 

□ 

□ 

□ 

□ 

Written, oral dissemination of 
results (publications, presentations) 

□ 

□ 

□ 

□ 
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C2. Do you agree or disagree with the following aspects of hosting this IRFP postdoctoral fellow? Mark 
one answer in each row 



Strongly 

disagree 

Disagree 

Agree 

Strongly 

agree 

Does 

not 

apply or 
do not 
recall 

The postdoctoral fellow integrated with 
staff/members of my research group 

□ 

□ 

□ 

□ 

□ 

The fellow had sufficient knowledge and 
expertise to be a full participant in a research 
collaboration with me 

□ 

□ 

□ 

□ 

□ 

The amount of time for the postdoctoral 
fellowship experience was too short 

□ 

□ 

□ 

□ 

□ 

Language was a barrier to the fellow's ability 
to interact with me and/or my group 

□ 

□ 

□ 

□ 

□ 

The fellow was willing to take appropriate risks 
necessary for research 

□ 

□ 

□ 

□ 

□ 

The fellow exercised appropriate caution in 
his/her approach to research 

□ 

□ 

□ 

□ 

□ 

The fellow spent sufficient time working on 
the proposed project 

□ 

□ 

□ 

□ 

□ 

Scientific cultures of our countries are similar, 
making productive collaboration possible 

□ 

□ 

□ 

□ 

□ 

Lifestyles and general cultures of our countries 
are a barrier to collaboration 

□ 

□ 

□ 

□ 

□ 


IF B7=NO, SKIP TO El 
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Module D: Outcomes of hosting an IRFP Postdoctoral Fellow 

Dl. Since the conclusion of this fellow's postdoctoral fellowship with you, have you collaborated or 
communicated further with this former postdoctoral fellow? Mark one answer 

□ I have collaborated further with this individual on research -GO TO Dla 

□ I have communicated further with this individual, but we have not collaborated further-^ 
GOTO D2 

□ I have not communicated further with this individual GO TO D2 

Dla. How recently has the latest collaboration occurred? Mark one answer. 

□ Within the past 6 months 

□ Within the past 12 months 

□ 1-2 years ago 

□ 3 or more years ago 

Dlb. What was/is the nature of this collaboration? Check all that apply 

□ I have a position in this individual's institution (within the U.S.) 

□ This individual has a position at my institution (outside the U.S.) 

□ We have exchanged ideas, data, research results, or tools 

□ We have co-authored research papers together 

□ We have co-advised students together 

□ We have visited each other at our institutions 

□ Other: [Textbox, 300] 

GOTO D3a 

D2. Why do you no longer collaborate with this former postdoctoral fellow? Check all that apply 

□ Our research interests diverged 

□ One or both of us lacked funding needed to maintain collaboration 

□ Language differences have hindered further collaboration 

□ Political or cultural differences have hindered further collaboration 

□ Geographic distance has hindered further collaboration 

□ I did not think that further collaboration would be beneficial for me 

□ The postdoctoral fellow did not actively pursue or maintain further collaboration with me 

□ One (or both) of us is too busy with other projects 

□ Other - specify [textbox, 300] 

D3a. Are you currently collaborating with any of [Fellow Name]'s colleagues? Check all that apply 

□ With this individual's former faculty advisor 

□ With graduate students who work with this former postdoctoral fellow 

□ With postdoctoral fellows who work with this former postdoctoral fellow 

□ With other researchers who work with this former postdoctoral fellow 

□ With none of the above 

D3b. Are you currently collaborating with other U.S. researchers? Mark one answer 

□ Yes^GOTO D3d 

□ No^GOTO D4 
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D3d. Have your collaborations with any U.S. researchers resulted from your participation as host to an 
IRFP postdoctoral fellow? Mark one answer 

□ Yes 

□ No 

D4. Please indicate the number of the following which you published or produced in collaboration with 
this postdoctoral fellow [Fellow Name]: 


Total number 
published/ 
in-press with this 
postdoctoral fellow 



Peer-reviewed journal articles 


Peer-reviewed conference publications (e.g. 
abstracts, conference papers, posters) 


Patents, registered or pending 


Book chapter(s) (e.g., in edited volumes) 


D4a. In what year did you most recently publish a paper or register a patent with this fellow? (Enter 4- 
digit year): [Textbox, yyyy] 

RANGE: 1990-2010 
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Module E: Satisfaction with Hosting an IRFP Postdoctoral Fellow 

We will not reveal responses that you provide on any question in this survey to any of the fellows that 
you hosted in your institution. Neither will any data that can identify any specific fellow or host be 
shared with the sponsor of this survey (NSF). 

El. On a scale of 1-5, with 1 being "much less satisfied," and 5 being "much more satisfied," how 
satisfied were you overall with this IRFP postdoctoral Fellow compared to other postdoctoral 
fellows? Mark one answer. 


<D 

© 

© 

© 

© 

Much less 
satisfied than 1 
have been with 
other postdocs 

Somewhat less 
satisfied than 1 
have been with 
other postdocs 

Equally satisfied 

as 1 have been 
with other 
postdocs 

Somewhat more 
satisfied than 1 
have been with 
other postdocs 

Much more 
satisfied than 1 
have been with 
other postdocs 


If you cannot select a response because you have not worked with any other postdoctoral fellows, 
please check here: □ If Respondent checks this box no rating should be allowed 


Ela. What did you find most satisfying or unsatisfying about this fellow or about the IRFP postdoctoral 
fellowship program as a whole? [Textbox, 2500] 


E2. Which of the following benefits (if any) did you derive as host to one or more IRFP postdoctoral 
fellows from the U.S.? Check all that apply 

□ Flosting an IRFP postdoctoral fellow was the first time I had collaborated with a researcher 
trained in the U.S. 

□ I established or renewed collaboration(s) with other US researchers 

□ I published (or researchers in my group published) papers based on work with this fellow 

□ I obtained (or researchers in my group obtained) funding based on the collaborative work 

□ I gave (or researchers in my group gave) one or more conference presentations based on 
the collaborative work 

□ I learned (or researchers in my group learned) new methodological/analytical techniques or 
theoretical approaches 

□ I became (or researchers in my group became) more familiar with the research enterprise in 
the United States 

□ Improved my ability to communicate in English 

□ Hosting an IRFP postdoctoral fellow changed the direction of some research projects in my 
group 

□ Hosting an IRFP postdoctoral fellow enhanced my interest in collaborating with US 
researchers 

□ Hosting an IRFP postdoctoral fellow made other US researchers more interested in 
collaborating with me or members of my research group 

□ None of the above 
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E3. Did you encounter any of the following challenges when hosting an IRFP postdoctoral fellow? 
Check all that apply 

□ The fellow's lack of familiarity with the primary language spoken in my research group made 
collaboration more difficult than anticipated 

□ The fellow and I had differences of opinion about the direction of research 

□ The fellow had unanticipated gaps in his/her preparation to conduct research with me 

□ The fellow did not devote enough time/effort to the research collaboration 

□ The fellow lacked sufficient understanding of cultural norms in my country 

□ The fellow was disrespectful, caused conflict within my research group 

□ The fellow needed too much guidance 

□ The fellow worked too independently, did not work well as a collaborator or team member 

□ Other challenges; please specify: [Textbox, 1000] 

□ None of the above 


E4. Have any of the following taken place at your institution as a result of your participation as host to 
an IRFP Fellow? Check all that apply 

□ My colleagues increased their own collaborations with US researchers 

□ Administration in my department (or research unit) became more supportive of collaboration 
with US researchers 

□ Additional policies, procedures, or structures have been put in place at my institution to 
facilitate international collaboration 

□ Collaborations with US researchers helped attract students and other researchers to my 
institution 

□ Other changes at my institution as a result of hosting IRFP Postdoctoral Fellow (please describe): 
[textbox, 1500] 

□ None of the above 

E4a. What were the best aspects of your affiliation with this IRFP fellow? [Textbox, 2500] 

E5. What were the most challenging aspects in your affiliation with this IRFP fellow? [Textbox, 2500] 

E6. Were there any unexpected outcomes in your affiliation with this IRFP fellow? [Textbox, 2500] 

E7. Would you recommend (or have you recommended) to others that they host an IRFP postdoctoral 

fellow from the United States? Mark one answer 

□ Yes, I would recommend (or I have recommended) to others that they host an IRFP postdoctoral 
fellow from the United States. GO TO E7a 

□ No, I would not recommend to others that they host an IRFP postdoctoral fellow from the 
United States. GO TO E7b 

□ I might recommend hosting an IRFP fellow, but it depends on the qualifications of the individual 
postdoctoral candidate GO TO E7d 

□ I might recommend hosting a postdoctoral fellow from the United States, but the IRFP program 
created challenges for me or my postdoctoral fellow. GO TO E7c 

□ I am not sure. GO TO E7d 


E7a. Why would you recommend hosting an IRFP postdoctoral fellow? [Textbox, 2500] Go to E7d 


Host Survey 


16 


E7b. Why would you not recommend hosting an IRFP postdoctoral fellow? [Textbox, 2500]? Go to E7d 

E7c. What challenges did the IRFP program present? [Textbox, 2500]? GO TO E7d 

E7d. Based on your experience hosting one or more postdoctoral fellows from the United States, 

what recommendations would you make to U.S. researchers about working with researchers in 
your country? [Textbox, 2500] 

E8. Please feel free to share any additional thoughts or recommendations about the IRFP program for 
postdoctoral fellows. [Textbox, 2500] 


SUBMIT button takes respondent to Thank you screen. 
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EXIT SCREEN: 

If we have identified the wrong respondent or have erroneous information, display this screen and 
FLAG this respondent for Abt follow-up (weekly basis) 


IRFP host survey 


Please accept our apology. 

The information you supplied suggests that you are not eligible to participate in this study or that we 
have reached you in error. We regret any inconvenience to you. If you have any questions about this 
study or you would like to make a comment, please contact one of the following individuals: 

■ Dr. Alina Martinez, Study Director at Abt Associates: IRFP surveygDabtassoc.com or (866) 421- 
6223 (may incur telephone charges if initiated outside the U.S.); 

■ John Tsapogas, Office of International Science and Engineering, NSF: itsapoga(5> nsf.gov . 

May we have permission to contact you by email in order to clarify your responses here? Answering 
"yes" does not obligate you to answer any questions. 

□ Yes, you may contact me. My preferred email address is: [textbox 150] 

□ No, please do not contact me. 

Thank you very much. 


THANK YOU SCREEN: 


IRFP host survey 


Conclusion 

Thank you for completing this survey. If you have any general comments about the survey, please write 
them below. 

Textbox, 2500 


If you have any questions about this survey or the study, please contact Dr. Alina Martinez, Study 
Director, Abt Associates, at (866) 421-6223 (may incur telephone charges if initiated outside the U.S.) or 
email her at IRFP survey(S)abtassoc.com . You may also contact John Tsapogas, who is overseeing this 
study at NSF, with any questions or comments: itsapoga (5> nsf.gov . 

Thank you for your assistance. We greatly appreciate your time and consideration. 
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Appendix F: Comparison of IRFP Applicants to SDR 


A comparison of all IRFP applicants to the Survey of Doctoral Recipients (SDR) provides a context for 
understanding the career outcomes of the pool of IRFP applicants relative to a nationally 
representative sample. 

The SDR is a longitudinal survey of a nationally representative sample of science, engineering, and 
health (SEM) doctorate recipients. Details of the sampling frame and how the SDR data (2006 and 
2008 waves) and the analyses conducted are provided in Appendix D. 

Given the overall purpose of the SDR (to describe general characteristics of doctoral education and 
early post-graduation employment for doctoral recipients), it was not designed specifically to 
address questions that might be important for the IRFP evaluation, including, for example, whether 
U.S. doctorate recipients in STEM fields co-author research publications with foreign collaborators. 
The only SDR question related to that type of outcome asked SDR respondents whether they had 
worked with individuals in countries outside the U.S. Other survey questions address more general 
career outcomes, specifically broad measures of employment, research productivity, and faculty 
rank and tenure for those working at institutions of higher education (IH Es). 

Findings should be interpreted with an understanding that there may be uncontrolled initial 
differences between those who applied to IRFP and who responded to the SDR. Comparisons 
between survey data from the SDR and from IRFP fellows are descriptive in nature. The analyses 
reported in this appendix can take into account (that is, statistically control for) the amount of time 
that has elapsed between when respondents earned their degrees and completed a survey, but the 
analyses cannot account for differences in prevailing conditions at the time data were collected 
from IRFP applicants and SDR respondents; specifically, there may well be factors related to 
economic conditions and employment that differ between the years during which the SDR 
respondents completed surveys (2006 and 2008), and when IRFP fellows completed surveys in 2011. 
Also, the SDR excluded individuals living outside the U.S., whereas the IRFP did not. Finally, because 
it was not possible to identify individual IRFP applicants in the SDR sample there could be overlap 
between the samples that renders the groups non-independent, but this overlap would be relatively 
small. 

Comparisons between SDR respondents and IRFP applicants were limited to SDR respondents who 
had completed a doctoral degree by the reference date specific to that SDR wave (April 1, 2006 in 
SDR 2006; October 1, 2008 in SDR 2008); to IRFP applicants who had applied for IRFP prior to 2008 
(N=950; 379 awarded and 571 unfunded). In addition, analyses controlled for gender, whether or 
not an individual was a member of an ethnic or minority group traditionally under-represented in 
STEM fields (URM), field of study for the first doctorate, and the number of years between receipt of 
first doctorate and the reporting year of outcomes. 


Findings: IRFP Applicants vs. SDR Respondents 


The comparison of estimates on career outcomes of IRFP applicants to the national estimates 
suggests that not only are they more likely to be engaged in international collaborations, they also 
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are a qualified pool of researchers, who go on to successful careers. IRFP applicants (including both 
funded fellows and unfunded applicants) were more likely to be engaged in international 
collaborations. IRFP applicants were also more likely to work in their current job with individuals in 
countries other than the U.S. (66 percent) than the typical STEM doctoral recipient (33 percent), a 
large and statistically significant difference (Exhibit F.l). Also, IRFP applicants reported having 
statistically significantly more publications and patents than SDR respondents (11 and 5 
publications, respectively, no exhibit). 24 

Former IRFP applicants had held, on average, a greater number of postdoctoral fellowships (1.6) 
than U.S. science and engineering doctoral recipients as a whole (.58), a statistically significant 
difference (no exhibit). 25 This finding is not surprising: by virtue of having applied to the IRFP 
program, these individuals were seeking a postdoctoral appointment, and the fact that the IRFP 
program supports an international postdoctoral appointment is likely to contributing less to this 
difference than is the difference between STEM doctorates who pursue a postdoc (domestic or 
otherwise) and those who do not. 

Exhibit F.l illustrates the differences in employment circumstances between former IRFP applicants 
and the SDR sample. Although most former IRFP applicants and most STEM doctoral recipients in 
the U.S. were employed during a specified reference week, former IRFP applicants were more likely 
to be employed (97 percent) than the average STEM doctorate (92 percent), the difference of 5 
percentage points was statistically significant. Statistically significant differences were also seen in 
the employment sectors. Specifically, former applicants were much more likely to be employed by 
an educational institution. Former applicants were less likely than the average STEM doctoral 
recipients in the U.S. to work in the private sector, local or state government or to be self-employed 
or a business owner. Former applicants were more likely to work for a type of employer other than 
those listed. 


Statistical models controlled for the different time periods about which respondents reported 
publications. 

The measure of the number of postdocs for IRFP fellows included the IRFP postdoctoral fellowship. For 
example, a fellow reporting 2 postdocs had one postdoctoral fellowship in addition to his/her IRFP 
postdoc. For comparisons of the number of postdocs held since receipt of first doctoral degree, the SDR 
2006 wave of data were used as these data were not available in the 2008 wave (IRFP N=950 Missing=0; 
SDR N=30,802 Missing=15). 
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Exhibit F.l: Characteristics of Employment of IRFP Applicants and SDR Respondents 


20 


Currently employed’ 1 
Employed at educational institution*** 
Private Sector*** 
Self-Employed or Business Owner*** 
Federal Government or Military Service 

Local or State Government* 

In current job, works with individuals 
in countries other than U.S.*** 


40 


60 


80 


100 



Former IRFP Applicants 
SDR Sample 


*p<.05, **p<.01, ***p<.001 

EXHIBIT READS: 96.6 percent of former IRFP applicants were currently employed during the reference week (October 1, 
2010) compared to 91.6 percent of SDR respondents (whose reference week was October 1, 2008). This difference of 5.1 
percentage points was statistically significant (p<.001). 

Currently employed: This item was answered by former IRFP applicants (N=950 Missing=0) and by SDR 2008 respondents 
who had completed a PhD by October 1, 2008 (N=29,974 Missing=0). 

Private Sector, Self-Employed, Federal Government or Military Service, Local/State Government: This item was answered by 
all former IRFP applicants who were employed as of October 1, 2010 in a non-postdoctoral position (N=723, Missing=0) 
and by SDR 2008 respondents who had completed a PhD by October 1, 2008 and who were employed during the week of 
October 1, 2008 (N=26,134, Missing=0). Items from which these data derive differed slightly between the IRFP Applicant 
Survey and the SDR 2008; thus. Local Government (city, county, school district) and State Government (including state 
colleges/universities) were combined into a single category for both groups; and U.S. Federal Government and U.S. 
MILITARY service, activity duty or Commissioned Corps (e.g., USPHS, NOAA) were combined for both groups. 

In current job, works with individuals in other countries: This item was answered by former IRFP applicants who were 
employed during the week of October 1, 2010 (N=912, Missing=0) and by SDR 2006 respondents who had completed a 
PhD by April 1, 2006 and were employed during the week of April 1, 2006 (N=27,119, Missing=0). This item was not 
included in the SDR 2008 wave. 

SOURCES: IRFP Applicant Survey-Items C9, Die, D3, and D4. SDR 2008-Items Al, All, and A12 and SDR 2006-Item A27. 


Among those individuals who were employed in educational institutions (other than a preschool, 
elementary or secondary school or school system), a greater percentage of former IRFP applicants 
had earned tenure (66 percent versus 54 percent, no exhibit), a statistically significant difference. . 
However, among respondents working in this type of educational institution, an equal percentage of 
former applicants and SDR recipients were in tenure track positions (e.g., assistant, associate, or full 
professor) (93 percent each, no exhibit). 


Among those individuals employed in educational institutions, respondents often reported that they 
occupied multiple roles with varied teaching, research, and administrative responsibilities (Exhibit 
F.2). A statistically significantly higher percentage of former IRFP applicants than SDR respondents 
reported that they held an academic position as a research faculty member, scientist, associate or 
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fellow (other positions could also be held concurrently) and held an academic position as a teaching 
faculty member (other positions could also be held concurrently). Other differences were not 
statistically significant. 


Exhibit F.2: Types of Academic Positions held (Solely or along with Another Position) by IRFP 
applicants versus SDR Respondents 


Research faculty, scientist, 
associate, fellow (solely or 
along with another position) 

Teaching faculty (solely or 
along with another position) 


Adjunct faculty (solely or 
along with another position) 


Other academic position (solely or 
along with another position) 


Dean, Department Head/Chair (solely or 
along with another position) 


20 


40 60 


80 100 



*p<.05, * **p<.01, ***p<.001 

EXHIBIT READS: Among those working at an educational institution, 60 percent of former IRFP applicants versus 44 
percent of SDR 2008 respondents held a research position (solely or concurrently with another position); ) this difference 
was statistically significant. 

NOTES: 

The academic positions are not mutually exclusive: Individuals could select more than one response. For example, 
individuals could hold a research faculty position and a teaching faculty position simultaneously. 

These items were answered by former IRFP applicants (fellows had to have completed their IRFP fellowship, including any 
U.S. -based "re-entry period," as of October 1, 2010) who were working at an educational institution (other than in a 
postdoctoral position) during the week of October 1, 2010 and who did not report working in a preschool, elementary, 
middle, or secondary school or system (N=551, Missing=0) and by SDR respondents who had completed a PhD by October 
1, 2008, who were working in an educational institution during the week of October 1, 2008, and who did not report 
working in a preschool, elementary, or secondary school system (N=ll,773, Missing=0). 

SOURCES: IRFP Applicant Survey-Item C9, Die, D2, D2a and D2b. SDR Survey 2008-Item Al, A12 and A14. 
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